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Foreword by The Series Editor 


Bio-Inspired Information Pathways: From Neuroscience to Neurotronics 


The human brain is the ultimate product of millions of years of evolution of Life. It is 
also an ultimate inspiration for the development of new technologies and an ultimate 
target to use these technologies to better understand the brain (reverse engineering) 
and beyond that—to improve Life on this planet by applying these technologies 
across disciplines. These technologies, along with the process of creating them, are 
referred here as Neurotronics. 

Neurotronics technologies are characterized by low power consumption, ultra-fast 
performance, and high density integration. Major research in the Neurotronics area 
is currently devoted to memristor-based devices and their applications for neuromor- 
phic computation (computation inspired by the information processing in nervous 
systems). 

For readers not familiar with the concept of memristor, but who want to read 
and understand the book, a memristor is defined as a non-volatile (not depending 
on power) electronic memory device that is based on a resistor which can manifest 
memory in terms of retaining its resistance after the current stimuli and changing it 
based on its previous values. It often is resembled to a synapsis in the human brain. 
The theoretical concept was first introduced by Leon Chua in 1971 as the fourth 
fundamental two-terminal circuit element, following the resistor, the capacitor, and 
the inductor. He called his paper “Memristor-the missing circuit element’. The first 
memristor circuit was physically realized, though 40 years after the introduction of 
the concept. 

And now neuroscientists and electronic circuit designers need to address the 
following challenges: 


(1) How a memristor circuit can be physically realized using different and more 
efficient physical and chemistry nano-properties of materials? 
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(2) How useful will these Neurotronics devices be to model the whole human brain 
at its actual size and functions? 

(3) What problems can be solved with these Neurotronics devices that cannot be 
efficiently solved with traditional neural networks and AI systems? 

(4) Can memristors, being in one of two non-volatile states (no power supply 
needed) of low- or high-resistance state, substitute totally the current transis- 
tors in the contemporary computers, saving vital power energy for the next 
generations of societies to come? 

(5) What is the relationship between memristor computation and nano-particle 
quantum processes? 

(6) How can current brain-inspired spiking neural network models be efficiently 
implemented on memristor devices? 

(7) How can memristor devices reveal back biological and neurophysiological func- 
tions in the human brain and the nervous system that cannot be revealed with 
the use of the current AI technologies? 

(8) What is beyond memristors, neuromorphic computation, and quantum 
computing? 

The book covers three major areas, leading from neuroscience to the present and 

future computer devices, and addressing some of the above questions: 


(1) Neuronal Architectures and Information Processing. 
(2) Bio-inspired System Technology. 
(3) Physical Substrates for Neuromorphic Engineering. 


Readers of the book can find the state of the art of a wide range of topics that are 
results of deep interdisciplinary studies involving knowledge from neuroscience, 
biology, psychology, physics, chemistry, computer science, electrical engineering, 
material science, network science, mathematics, and nonlinear dynamics. 

Neuroscientists can learn more about topics, such as neuronal synchrony; self- 
organized criticality; plasticity; connectomics; nervous system growth under external 
stimuli; fundamental biological information pathways that explore the spatiotem- 
poral fundamentals of nervous systems; homeostasis; local versus global aspects 
of information processing in nervous systems, mimicking neuroplasticity with 
memristor devices; perceptual decision-making circuit; modeling the biological 
process of pattern recognition in the box jellyfish nervous system; and many more. 

Electronic engineers, computer scientists, and information technologists can 
learn more about cutting-edge technology, including memristive and memsensor 
devices; memristive properties, such as J-V linearity, switching time, retention, 
state numbers, time dependency, device variability; different technologies of bi- 
layer metal oxide memristive devices, floating-gate transistors as memristors, ex situ 
versus in situ measurements of memristive devices, silver-based filamentary memris- 
tive devices, micro-electro-mechanical systems (MEMS), bio-inspired, neuromor- 
phic acoustic sensing, application-specific integrated circuit (ASIC) technologies, 
chemical oscillators, neuromorphic hardware systems, and many more. 
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The book is a big step forward toward the development of new neuromorphic 
computer systems that process not only vector-based and bit-represented information, 
but create deep information spatiotemporal pathways, similar to the pathways created 


in the human brain when learning and performing physical and cognitive tasks, as in 
the example shown in the Fig. 1. 


Prefrontal cortex 


Fig. 1 Example of a deep information pathway for learning and processing visual stimuli in humans 
for image classification. Location of cortical areas: V1 = primary visual cortex, V2 = secondary 
visual cortex, V4 = quaternary visual cortex, IT = inferotemporal cortex, PFC = prefrontal cortex, 
PMC = premotor cortex, MC = motor cortex (from Kasabov, N., Time-Space, Spiking Neural 


Networks and Brain-Inspired Artificial Intelligence, Springer Nature (2019) 750p., https://www. 
springer.com/gp/book/9783662577134) 


This is a promising step toward the realization of a true model of the human brain 
and various brain-inspired cognitive and robotic systems. 

As the Springer Editor of the series of Bio-and Neuro-systems, I am delighted 
to welcome this book to our series as it incorporates the state of the art of biology, 


neuroscience, physics, chemistry, and their impact on the development of novel 
information technologies. 


This book represents the results of many years of collective work of leading 
European institutions under a large project (CRC1461) titled Neurotronics: Bio- 
Inspired Information Pathways (https://www.crc1461-neurotronics.de/research/pro 
ject-overview) and I would like to encourage other research groups to follow this 
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example and to share and disseminate their results and knowledge from their projects 
in our Springer series of Bio-/Neuro-systems. 

Prof. Nikola Kasabov 

Life Fellow IEEE, Fellow RSNZ 

Fellow INNS College of Fellows 

Penrose, New Zealand 


Preface 


Nature is a tinkerer, not an inventor 


François Jacob 


We are delighted to present this volume which addresses basic and likewise fasci- 
nating aspects of information pathways in nervous systems and their undisputed 
relevance for future computing architectures. 

Indeed, creatures are very well adapted to their specific ecological niche, resulting 
from a billion years of ongoing evolution and the therewith associated interaction of 
creatures with their environment during their life span. From an engineering point of 
view, nervous systems process information in such a way that silicon technology, the 
holy grail of modern digital computing strategies, seems to be outstripped. Electronic 
devices and circuits, such as transistors, memories, and processors, are optimized for 
small parameter spreads to run at GHz clock frequencies, applying precise pulse 
timing. In particular, they exploit nanosecond signal pulses that travel nearly at 
the speed of light along well-ordered transmission lines interconnecting the system 
parts in an essential two-dimensional topology. In contrast, information pathways in 
nervous systems are characterized by highly irregular tissue consisting of neurons, 
synapses, and axons. Low conduction velocities of the order of several m/s lead 
to pronounced signal retardation, i.e., delay. Nervous systems can be considered 
as time-varying networks in which spike dynamics and cellular morphology are 
intricately linked and reciprocally interwoven. Understanding how living creatures 
perform pattern recognition and cognitive tasks could be a particularly important 
ability for future computing architectures. 

Understanding how living creatures perform pattern recognition and cognitive 
tasks is of particular importance for computing architectures: by studying these infor- 
mation pathways refined over eons of evolution, researchers may be able to stream- 
line the process of developing more highly advanced, energy-efficient autonomous 
systems. With the advent of novel electronic and ionic components along with a 
deeper understanding of information pathways in living species, a plethora of oppor- 
tunities to develop completely novel information processing avenues are within 
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reach. This book encompasses research works on fundamental studies on memris- 
tive compartments, such as mesa-type devices and self-assembled nanostructures to 
neuromorphic circuits. The latter comprises units for pattern recognition, decision- 
making in order to mimic basal synaptic functionalities up to neuromorphic systems 
for pattern recognition, decision-making, and bio-inspired sensing. Here for guide- 
lines are basal information pathways in nervous systems, from the local synaptic 
level to the entire nervous system network. The dual importance of local learning 
rules is addressed, from spike timing dependent plasticity at the neuron level to the 
interwoven morphological and dynamical mechanisms of the global network. Biolog- 
ical principles are highlighted, including phylogenies, ontogenesis, and homeostasis, 
with particular emphasis on network topology and dynamics and their relevance for 
engineered pulsed-coupled oscillators ensembles. 

This volume summarizes research work performed within the Research Unit 2093, 
entitled: Memristive Devices for Neural Systems between 2014 and 2020, and within 
the in 2020 established Collaborative Research Center 1461: Neurotronics-Bio- 
Inspired Information Pathways. Both projects are funded by the German Research 
Society (DFG). 
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Abstract As a result of a hundred million years of evolution, living animals have 
adapted extremely well to their ecological niche. Such adaptation implies species- 
specific interactions with their immediate environment by processing sensory cues 
and responding with appropriate behavior. Understanding how living creatures per- 
form pattern recognition and cognitive tasks is of particular importance for computing 
architectures: by studying these information pathways refined over eons of evolu- 
tion, researchers may be able to streamline the process of developing more highly 
advanced, energy efficient autonomous systems. With the advent of novel electronic 
and ionic components along with a deeper understanding of information pathways 
in living species, a plethora of opportunities to develop completely novel informa- 
tion processing avenues are within reach. Here, we describe the basal information 
pathways in nervous systems, from the local neuron level to the entire nervous sys- 
tem network. The dual importance of local learning rules is addressed, from spike 
timing dependent plasticity at the neuron level to the interwoven morphological and 
dynamical mechanisms of the global network. Basal biological principles are high- 
lighted, including phylogenies, ontogenesis, and homeostasis, with particular empha- 
sis on network topology and dynamics. While in machine learning system training 
is performed on virgin networks without any a priori knowledge, the approach pro- 
posed here distinguishes itself unambiguously by employing growth mechanisms as 
a guideline to design novel computing architectures. Including fundamental biolog- 
ical information pathways that explore the spatiotemporal fundamentals of nervous 
systems has untapped potential for the development of entirely novel information 
processing systems. Finally, a benchmark for neuromorphic systems is suggested. 


Keywords Bio-inspired Computing - Phylogenesis - Ontogenesis - Homeostasis + 
Artificial spatio-temporal networks 
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1 Introduction 


Is it truly possible to implement higher brain functions, such as perception or con- 
sciousness, in engineered systems? This question has been frequently raised over 
the last few decades and has led to distinct views over time, as both neurobiological 
understanding and available computational capabilities advanced [1-7]. The essence 
of this question goes back to the fundamental relation between matter and mind, 
which was addressed as early as ancient Greece, and emerged in the principle of 
“Dualism” most famously defended by the philosopher René Descartes in the six- 
teen century [8]. Descartes postulated that the body (matter) and the mind are distinct 
and separate units in human beings because he could not imagine that mental phe- 
nomena could be explained by natural mechanisms [9]. However, the invention of 
electroencephalography (EEG) and imaging techniques, such as functional magnetic 
resonance imaging (fMRI), enabled the study of inner information processing in the 
human brain and individuals’ states of consciousness [10-13]. As a result, the strict 
distinction between matter and mind has become blurry [14, 15]. Strong evidence 
has been found that the inner representation of the human brain (the mind) is related 
to its neurochemistry (the matter), e.g. the amount and type of neurotransmitters 
and/or drugs within the nervous system [16, 17]. It is therefore worthwhile to recon- 
sider the relationship between mind and matter when engineering artificial systems 
to exhibit higher brain functions by considering recent progress in nanoelectronics 
and neurobiology. 

This perspective on future computing is motivated by three key aspects. First by 
the recent, growing movement to reboot the entire field of computing, i.e. how data 
are processed. Second by state-of-the-art, fundamental progress in neurosciences, 
including the fields of complex networks and dynamic brain states. Third by advances 
in materials science and nanoelectronics that have led to, e.g., memristive devices, 
nanoparticle/nanowire networks, and fluidic memristors, providing new functionality 
in electronics, such as synaptic-like plasticity or spatio-temporal networks [18-22]. 
With the foreseen restrictions on current digital computing, the question “What comes 
next?” finds its answer in merging novel discoveries made on the nervous system’s 
information pathways with the development of novel electronic devices, paving the 
way to an entirely new kind of computing. 

In this perspective, we present a concept of an artificial spatio-temporal network 
which uses temporal and structural mechanisms in nervous systems as guidelines. It 
addresses the important, interwoven spatiotemporal aspects of information pathways 
and processing in nervous systems [23-28]. The state of nervous system criticality 
combined with the blooming and pruning of nervous cells during growth might be an 
interesting guideline to develop new computing principles [23, 25, 29-34]. Compo- 
nents essential for artificial spatio-temporal networks and a pathway to realize it, are 
presented, including biological fundamentals such as phylogenies, ontogenesis, and 
homeostasis [35, 36]. However, these basal biological mechanisms alone might not 
be sufficient to establish mental functions in artificial systems; we therefore include 
the temporal binding hypothesis developed in neuroscience as a further essential 
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guideline [10, 37-40]. The synchronized firing of neural ensembles across different 
brain regions is treated as a fundamental neural mechanism that defines how a hier- 
archical network structure, such as the brain, can integrate several sensory inputs to 
determine the unity of an object (for example linking form, color, size, motion, etc., 
together) [41, 42]. Opportunities and possible limitations of this approach towards 
implementing higher brain functions in artificial systems, such as perception and 
consciousness, will be discussed. The paper is arranged as follows: 

In chapter “Neuromorphic Circuits with Redox-Based Memristive Devices”, the 
current status of computing architectures is summarized. In chapter “Redox-Based 
Bi-Layer Metal Oxide Memristive Devices”, we present a condensed overview of 
advanced device components, with a focus on memristive switching devices. We sub- 
sequently address spatiotemporal information processing in nervous systems, includ- 
ing network structure, network dynamics, and homeostasis in chapter “MemFlash— 
Floating Gate Transistors as Memristors”. An artificial spatio-temporal network 
concept is introduced in chapter “Critical Discussion of Ex situ and In situ TEM 
Measurements on Memristive Devices”, where we hypothesize on which informa- 
tion pathways might lead to higher brain functions in engineered systems based on 
hardware-oriented electrochemical electronics but also discuss current limitations 
of this approach. In chapter “Modeling and Simulation of Silver-Based Filamen- 
tary Memristive Devices” a possible benchmark is discussed for bio-inspired sys- 
tems. Chapter “Integration of Memristive Devices into a 130 nm CMOS Baseline 
Technology” provides a discussion on the practical implementation of an artificial 
spatio-temporal network, to mimic basal biological information pathways. 


2 The Current State of Information Technology 


The sixties marked the beginning of a glorious time in information technology as the 
tremendous opportunities of silicon technology merged with the concept of Boolean 
computing, resulting in the first digital revolution [43]. 

This development followed the exponential increase over time of electronic com- 
ponents integration on a chip predicted by Gordon E. Moore, combined with a sequen- 
tial data processing architecture comprising a central processing unit and memory 
for data storage, for which Alain Turing and John von Neumann laid the founda- 
tion years before [44—46]. The tremendous technological and economical success 
of the digital revolution is still going strong today with seemingly no end in sight. 
Billions of transistors on a single processor chip, displaying features as small as 
about 10 nm in size and clock frequencies of a few GHz, are the current standard 
in CMOS (Complementary Metal Oxide Semiconductor) technology, representing 
the backbone of today’s semiconductor industry [47—49]. However, during the last 
couple of years, dark clouds have appeared on the horizon for the semiconductor 
industry. The envisioned goal of downscaling devices with every new circuit gen- 
eration to the nm level has created an ongoing need to develop ever more sophis- 
ticated and expensive fabrication tools for e.g. lithography, dry-etching, and layer 
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deposition [50-52]. As a result, each new circuit generation entails an increasing 
economic risk for semiconductor companies. Moreover, over the last few decades, 
progress in processor core clock rates have overtaken memory access and access 
times, leading to a cumbersome situation where data transmission between the arith- 
metic logic unit (ALU) and memories dominates instead of the arithmetic informa- 
tion process itself. This system level-related challenge is called memory latency (or 
memory gap) and is a consequence of the von Neumann bottleneck, where data is 
processed sequentially [47, 53, 54]. Two major obstacles restrict the further devel- 
opment of information technology, namely limitations in downscaling at the device 
level, and memory latency on the architecture. Although society is experiencing 
a second digital revolution via the resurgence of artificial intelligence (AI) and the 
Internet of Things (IOT), Moore’s law, which has been driving the computer industry 
for decades, is becoming outdated as the limits of device integration and/or econom- 
ical boundaries have now been reached. The incredible advances made by the first 
digital revolution based on binary “0” and “1” computation combined with the latest 
achievements in the field of machine learning led to great progress in speech and 
pattern recognition, while rendering autonomous driving tangible. Yet, additional 
challenges are growing increasingly problematic behind the scenes. Huge, power 
consuming hardware systems in the form of cloud servers are now mandatory to 
support recent advancements in AI and the IOT. This is why global digital players, 
such as Google, Amazon, and Facebook, as well as bitcoin trading platforms need 
energy-hungry server farms [55]. 

On the system level, and in particular since the advent of the internet and the 
renewed interest in AI, the power consumption of the digital world is growing without 
limits, in increasing conflict with sustainable and climate-neutral resource manage- 
ment. Moreover, future autonomous electric vehicles require both high recognition 
capability and low power consumption. It therefore is hardly surprising that the 
semiconductor world is currently in an era of upheaval, turning a new page on infor- 
mation processing based on novel computing architectures and advanced hardware 
components. 


3 Advanced Computing Architectures and Novel 
Electronic Devices 


The aim of this section is to give a short survey on novel computing architectures and 
advanced electronic devices. We do not intend to present a comprehensive overview 
but instead to give a taste of the developments currently being pursued to overcome the 
limitations of digital computing and to establish new computing primitives. To sim- 
plify access to the different research areas for interested readers, we discuss seminal 
and overview papers and present recently published pioneering research. Nonethe- 
less, we are aware that the given reference list is far from exhaustive. In addition, 
this section is critical to understanding the similarities, and most importantly the 
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Fig. 1 A shamrock-like VIN 
illustration of the three 


development areas, which /wechine EN > 
characterize the currently d Neuromorphic 


expansive development in Engineering 
the field of Artificial 


Intelligence (AI) 


distinctions, between artificial spatio-temporal networks and standard neuromorphic 
computation presented in Sect. 5. While traditional von Neumann computing con- 
tinues to dominate the ICT scene, recent groundbreaking innovations in alternative 
computing architectures and advanced electronic devices have become hard to ignore 
[22, 56—60]. 

These developments are threefold. Firstly, somewhat older concepts, such as arti- 
ficial neural networks (ANN) leading to Deep Learning (DL) systems, have received 
an impressive performance boost through novel and efficient algorithms paired with 
more powerful electronics hardware [61]. Secondly, new technologies, such as Quan- 
tum Computing and Reservoir Computing (RC), have appeared, leading to remark- 
able results [62—64]. Thirdly, in the field of nanoelectronics, a plethora of advanced 
device structures and novel functional components has led to a rethink of traditional 
computing architecture, paving the way to in-memory computing that circumvents 
the von-Neumann bottleneck [58, 60, 65, 66]. In Fig. 1 a shamrock-like illustration 
highlights these three research areas. 

The first leaf representing Machine Learning encompasses Artificial Neural Net- 
works (ANN), Spiking Neural Networks (SNN), Reservoir Computing (RC), Long 
Short Term Memory (LSTM), and Deep Learning (DL) systems [67]. The foun- 
dations of Neural Networks were laid by McCulloch and Pitts [68], Rosenblatt’ s 
Perceptron [69] for ANNs, and von Neumann’s postulate of SNNs in 1956. More 
recent inventions from Jäger (Reservoir Computing) [70], Hochreiter and Schimd- 
huber (Long-/Short-Term Memory) [71]) and Hinton (DL) [67]) have advanced the 
field one huge step forward and comprise the backbone of today’s AI. 
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In the second leaf, the field of neuromorphic engineering, initiated by Carver 
Mead and Mohawa and Rodney Douglas, seeks to mimic the basal mechanisms 
of information processing in nervous systems via an essentially hardware-oriented 
approach [72-74]. In recent years, great progress has been made in the development 
of bio-inspired processors. Here, event-based spiking neural networks (SNNs) in the 
form of either mixed (analog and digital) or strictly digital signal processing provides 
novel opportunities for low-power data processing [59, 60, 75-77]. Interestingly, 
some of the spiking neuromorphic circuits work at biologically relevant frequencies, 
exhibiting low energy consumption. One point of merit for neuromorphic engineering 
is their energy per synaptic operation (SOP), which is in the pJ to nJ range for 
neuroprocessors [57, 60, 77]. 

Hence, the incorporation of relatively few basal mechanisms of biological infor- 
mation processing, such as leaky-integrated firing, axon delays, and local learning 
rules, can lead to significant improvements in resource management. 

Recent advances in the field of nanoelectronics devices, such as memristive 
devices, nanoparticle networks, nanowire networks, or memristive fluids, compose 
the third leaf of advanced computer architecture. Research in silicon nanoelectronics 
is dominated by the development of new field effect transistors (FET) [52, 78] for 
the next generation of CMOS circuits, as well as entirely novel devices and materials 
exhibiting advanced functionalities [66, 79-84]. In particular, the memristor (orig- 
inating from memory and resistor, also called memristive device) is a two terminal 
device that exhibits attractive features for various applications in the post-Moore area, 
generating considerable interest. Memristive devices were intensively studied in the 
sixties and seventies [85-87]. The field was further propelled forward by the estab- 
lishment of the theoretical background of memristors by Leon Chua (1971), with 
the corresponding experimental realization and interpretation by Hewlett-Packard 
(HP)-Labs (2008) [88, 89]. Over the years, numerous books and reviews have cov- 
ered fundamental and practical properties of memristive devices and their related 
circuits [18, 84, 90-92]. 

So far we have described nanoelectronic devices fabricated using top-down meth- 
ods, where the layers are deposited on an entire wafer and the devices are patterned by 
lithography and dry-etching [93—95]. In bottom-up approaches, functional materials 
are deposited or synthesized to obtain networks, such as irregular nanowires and/or 
3D textures. Often the self-assembly capabilities of materials are exploited to create 
complex structures. Top-down and bottom-up approaches are habitually combined to 
create the electrical connections necessary to characterize the structures [96]. In the 
context of bio-inspired computing, we would like to highlight here the work done on 
nanowire networks [20, 97—103]. The structure of such networks, and in particular 
their dynamic properties, reflect basal functionalities as observed in nervous systems, 
such as small-word connectivity and self-organized criticality (SOC) [23, 104]. We 
would like to emphasize that the three ICT research areas shown in Fig. 1 are not 
independent from one another: there is considerable overlap between them, which 
has proven to be mutually beneficial. 
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3.1 Advanced Computing Architectures 


Here we present a few concepts of novel and reconsidered computing architectures. 
We would like to emphasize that the following four examples were chosen to demon- 
strate the diversity of the field but are not intended to give a comprehensive overview. 
The icons in Fig. 2 represent different computing principles. 

For concepts other than those shown in Fig. 2, such as quantum computing, cellular 
automata, and probabilistic computing, we refer to the literature [62, 80, 105-107]. 
We focus on comparing today’s digital computing to in-memory computing, vector 
matrix multiplication, reservoir computing, oscillatory computing, and bio-inspired 
computing (see icons in Fig. 2). 

In order to overcome the von Neumann bottleneck of digital computing (Fig. 2a), 
near-memory computing was developed in 1990 [108]. Here, the strict separation 
of an arithmetic logic unit communicating with several distinct memories was elim- 
inated. Part of the computational tasks was performed within the memory itself, 
leading to more efficient computing. This development has recently shifted to a 
higher gear, leading to in-memory computing (Fig.2b) following the invention of 
memristive crossbar-arrays [58, 109]. Vector matrix multiplication (Fig. 2c) is con- 
sidered a key hardware booster in Deep Learning. The time and energy consuming 
task of vector matrix multiplication is performed in a memristive crossbar-array in 
which the input and output layer are interconnected by an array of weighted, check- 
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Fig. 2 The illustration shows snapshots of different computing strategies. a Digital computing, b 
in-memory computing, ¢ matrix multiplication, d reservoir computing, e oscillatory computing, f 
bio-inspired computing, g probabilistic computing, h cellular automata, i quantum computing 
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board arranged memristive devices [58, 92, 110]. Vector matrix multiplication is 
an example of how Deep Learning may benefit from the development of new elec- 
tronic devices, e.g. memristors. Reservoir Computing (Fig. 2b) was independently 
invented by Herbert Jager and Wolfgang Maass and belongs to the general frame- 
work of Recurrent Neural Networks (RNN) [70, 111]. In RNNs, a backpropagation 
through-time procedure is typically applied to adjust (train) the weights of the net- 
work to desired target functions. Here, a significant amount of time is required, 
with no certainty that the optimal weights will be set after learning. Thus in RC the 
reservoir consists of an ensemble of nonlinear elements coupled to one another. The 
reservoir projects incoming data and time series to a higher dimension that can be 
easily readout by conventional classifiers, in which the training is executed by means 
of a linear regression, for example. This reservoir can be either virtual or physical. 
These aforementioned reservoirs are designed like neural networks in which the con- 
nections are randomized but remain fixed during computation. Physical reservoirs 
are those which rely on natural systems exhibiting nonlinearity [64, 112, 113]. 

The goal of analog computing is to mimic complex technical systems by means 
of electronic circuits which represent key system parameters as a set of voltage lev- 
els at nodes. Oscillatory computing (Fig. 2e) refers to a subset of analog computing 
in which the oscillator frequencies and phases enrich the representation of infor- 
mation. Oscillatory systems are omnipresent in nature and engineering [114-116]. 
Technically, oscillators can be realized in numerous ways, such as in discrete or inte- 
grated semiconductor electronics, spin-torque devices, Josephson junctions, optical 
devices, or micro electro-mechanical systems(Schneider et al. 2018; Lequeux et al. 
2016; Chen et al. 2020; Ignatov et al. 2016; X. Cheng et al. 2021; Feldmann et al. 
2019a; C. Lenk, L. Seeber, and M. Ziegler 2020) [82, 117—122]. In general, dynam- 
ical systems and their coupled oscillators may offer elegant solutions to compute 
HP-hard problems. Coupled oscillator networks have been successfully exploited in 
the field of pattern recognition [123, 124]. However, larger systems have not yet been 
successfully developed due to noise-stability problems and device constraints in the 
new class of compact oscillators based on VO% or NbO,, for example [125, 126]. 

The term bio-inspired computing (Fig.2f) is only loosely defined. To a large 
extent, the computing primitives described above (see Figs. 1 and 2b—e) are more 
or less biologically motivated. The Perceptron is a crude blue print of a neuron and 
is still today at the heart of Deep Learning systems [69]. In-memory computing 
is a strategy to abrogate the strict separation of the ALU and memory in digital 
computing, and is derived from biological information processing where logic and 
memory are blended. Neuromorphic processors contain circuits that can execute 
the Leaky Integrate-and-Fire dynamics of neurons, including the biologically moti- 
vated winner-take-all (WTA) principle, and introduces axon delays [57, 127]. Cou- 
pled oscillators imitate the orchestra of neural ensembles, i.e. the communication of 
separate brain regions which is considered to be the fundamental mechanism that 
explains perception [10, 37, 41]. Cellular automata, for example, were introduced 
by John von Neumann to describe self-reproduction in biology [128]. Probabilis- 
tic computing is based on Bayesian inference, which is closely related to the way 
humans make decisions [129, 130]. Therefore, it is essential to declare precisely to 
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what extent an artificially built system is bio-inspired and which biological pathway 
have been applied as design principles [131]. 


3.2 Novel Electronic Devices 


There is an ongoing effort to shrink silicon FETs to feature sizes below 5 nm. The 
FinFET structure has dominated CMOS technology since its invention in 1989 [132]. 
Novel designs, such as GAAFET (Gate-All-Around) and MBCFET (Multi-Bridge- 
Channel), are serious candidates for next generation CMOS chips (see Fig. 3a) [52]. 
Aside from this ongoing improvement of conventional FETs, devices with novel 
functionalities and materials have been attracting considerable interest to implement 
novel computing architecture. Magnetic Josephson Junctions, photonic synapses, 
and bio-organic memories represent only a fraction of current development strategies 
[21, 80, 117, 121]. In Fig. 3b—d, unconventional nanoelectronics device structures 
are illustrated. In Fig. 3b, a memristive device structure is illustrated, comprising two 
electrodes separated by a memristive layer. In the same Figure, a qualitative I-V curve 
of a memristive device is shown alongside a sketch of a biological synapse (see also 
Fig. 4), highlighting that memristive devices are promising artificial synaptic coun- 
terparts due to their capability of presenting variable resistive weights in engineered 
neural networks [19]. 

One universal property of the memristive device concept is that the memristive 
state depends on previously induced charge flows, applied currents, or applied electric 
fields, thus storing a historically-determined resistance state. For details concerning 
resistive switching and the underlying physical-chemical mechanisms, we refer the 
reader to the references given in the figure caption (Fig.3) and the overwhelming 
literature on the subject [19, 91, 134, 135]. 
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Fig.3 Schematics of four advanced device components. a 3D view graph of a GAAFET as applied 
in today’s latest digital processors [52], b sketch of a memristive device including a qualitative I-V 
curve and illustration of a synapse [18, 91, 133], c cartoon of a 3D nanowire network [20, 22, 
81, 97, 99-102], d 3D cross-sectional graph of a fluidic memristive device adapted from [21] with 
permission 
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It is this concurrently complex and simple device concept, together with the 
tremendous predicted potential for breakthrough technologies in areas such as uni- 
versal memories and novel non-Boolean computing schemes for cognitive electronic 
systems, that propels the research and development of memristors and memristor- 
based circuits worldwide. It is important to mention that, in contrast to the theoreti- 
cally simplistic memristor concept, in practice the realization of memristive devices 
by modern thin film technology is a task littered with obstacles. Up until now, a 
huge number of experimental findings on memristor devices consisting of a broad 
variety of metal/insulator material combinations have been published, all of which 
show memristive I-V curves [91, 133, 135]. At first glance, it seems that the toolbox 
of resistive switching devices is ready for nearly any circuit application: simply pick 
a device concept and follow the extensive materials and methods laid out in the lit- 
erature. However, a closer look at the fine details casts a dark shadow on this bright 
research field, leading to a harsh awakening based on hard facts. These “hard facts” 
are the requirements and boundary conditions set by the envisaged circuit applica- 
tions, in which memristors must fit technologically, electronically, and economically. 
Currently, two main development avenues can be explored for memristive devices. 
The first focuses on resistive random access memories (RRAMs). It is believed that 
the zoo of today’s existing memory diversity can be replaced by a single (universal) 
memory concept. RRAMs are considered attractive candidates for universal memo- 
ries because they: (i) show non-volatile data storage, (ii) can be densely integrated, 
(iii) are fast, and (iv) are cheap to produce. In particular, such a universal memory 
might attenuate the problem known as memory latency in modern digital computers 
[54, 136]. Besides the RRAM goal which may be categorized under the label “More 
Than Moore”, novel and very appealing computer architectures have been proposed 
in which memristors might play a vital role. Another main focus of possible memris- 
tive device applications may be associated with such catchphrases as: non-Boolean 
computing, bio-inspired information processing, neuromorphic engineering, or cog- 
nitive electronics [66, 137—140]. On the local, synaptic level, learning in nervous 
systems is explained by Hebb’s learning rule and Spike-timing dependent plastic- 
ity (STDP), amongst others [141]. STDP and other memory-related mechanisms 
observed in nervous systems, such as Long-term Potentiation (LTP) and Long-term 
Depression (LTD) [142], have been successfully mimicked by memristive devices 
[143, 144]. Moreover, traditional studies known from behaviorism, such as classical 
conditioning (e.g. Pavlov’s dog), anticipation, and optical illusions, were success- 
fully realized experimentally by both single and pairs of memristive devices [119, 
145-148]. The extent to which larger networks of memristive devices are able to 
mimic higher brain functions is still unknown. 

In Fig. 3c, a sketch of a nanowire network (NWN) is shown. NWNs have been 
successfully synthesized for various materials, such as metals, oxides, and semicon- 
ductors [113, 139]. Nanowires show appealing features with respect to bio-inspired 
computing from the point of structure, topology, and inherent dynamics [99-102, 
149]. In recent comprehensive reviews by Zhu et al. and Kuncic and Nakayama, hall- 
marks known from biological systems as small-world connectivity (topology) and 
self-organized criticality (dynamic) were addressed [22, 101]. Interestingly, brain- 
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like avalanche effects have been observed in NWNs that exhibit dynamic features 
found in nervous systems [23, 25, 100, 103]. Finally, we would like to emphasize 
that emergent neuromorphic materials and devices are not restricted to the solid state 
phase. In Fig.3d, a sketch related to a nanofluidic device is shown. Bocquet and 
co-workers demonstrated by analysis and molecular dynamic simulations that ion 
transport across quasi-two-dimensional slits under an electric field displays memris- 
tive I-V curves, as well as spiking voltage patterns in accordance with the Hodkin— 
Huxley model of biological neurons [21, 150]. We would like to emphasizes that 
while these examples of NWNs and nanofluidics clearly demonstrate that the mate- 
rial “tool box” offers novel opportunities to implement higher brain function, its full 
potential has yet to be fully explored. 


4 Information Processing in Nervous Systems 


This perspective explores the role of information processing observed in nervous 
systems as a basis for the development of energy-efficient technological comput- 
ing systems, and even the possibility of implementing higher brain functions in 
engineered systems. Nervous systems offer paradigms to improve energy-efficient 
artificial information processing units. The exploration of signal pathways in nervous 
systems shows us how evolution led to extremely energy-efficient signal processing 
units (nervous systems). For example, the human brain dissipates a power of only 
roughly 20-25 W. This, in addition to the amazing capabilities of humans’ vision 
and hearing, reveals fascinating opportunities for autonomous vehicles or speech 
recognition. Hence, processing sensitive data in server clouds may lead to severe 
security concerns. The data of millions of cars in motion, including their controlla- 
bility, falling into the wrong hands could lead to fatal attacks; Local data processing 
in an autonomous car with low power consumption is preferable. 

Creatures are very well adapted to their specific ecological niche, a result of a 
hundred million years of ongoing evolution and the associated interaction between 
creatures and their environment throughout their life span [151—153]. In particular, 
information pathways in nervous systems are prototypes for engineers to perform 
cognitive tasks in quasi-real time with extremely low power consumption [154]. 
These features alone, and the information processing behind them, represent attrac- 
tive models for entirely new computing architectures. In Sects.4.1 and 4.2, local 
and global aspects of information processing mechanisms are presented, respec- 
tively. Distinct differences between digital computing and information pathways in 
biological systems are highlighted in the framework of topology and dynamics to 
motivate the concept of artificial spatio-temporal networks, as subset of the field 
of bio-inspired information processing. In Sects. 4.3 (Phylogenies and Ontogenesis) 
and 4.4 (Homeostasis), we underline important hallmarks of information processing 
in biological systems which have so far only been partly considered for artificial sys- 
tems. Note that in chapter “MemFlash—Floating Gate Transistors as Memristors’”’, 
we do not address how such mechanisms can be established in electronics: This is the 
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Fig. 4 Blueprint of a neuron including an enlarged sketch of a synapse and the illustration of a 
single action potential, a spike 


subject of chapter “Critical Discussion of Ex situ and In situ TEM Measurements on 
Memristive Devices”, where several approaches are proposed to implement an artifi- 
cial spatio-temporal network. It is not our goal to develop another pattern recognition 
system but to address the fundamental question: “To what extent can higher brain 
functions be reproduced in artificial systems?”. We believe that essential information 
pathways in biology have been to a large extent overlooked, as detailed in this per- 
spective. One important difference between artificial spatio-temporal networks and 
contemporary AI and neuromorphic engineering is that essential growth mechanisms 
observed in nervous systems are exploited as a guideline in the former. 


4.1 Local Aspects of Information Processing in Nervous 
Systems 


In contrast to current clock-driven Boolean Turing machines, information processing 
in biological nervous systems is characterized by highly parallel, energy-efficient, and 
adaptive architecture [53, 155, 156]. When it comes to pattern recognition, failure 
tolerance, and cognitive tasks, even simple creatures outperform supercomputers, 
in particular regarding power dissipation. Fundamental building blocks leading to 
such remarkable properties exploit neurons as central processing units, which are 
interconnected by synapses to form a complex dynamical three dimensional network, 
the connectome [157]. In Fig. 4, the structure of a neuron is sketched, including the 
soma, dendrites, the axon, and connections to other neurons by synapses. 

An action potential (spike) is defined as a sudden transitory and propagating 
change in the resting potential across amembrane. Action potentials sent from presy- 
naptic neurons are received via the dendrites and synapses of the postsynaptic neu- 
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rons. Those signals are integrated within the cell body of the postsynaptic neuron. 
When a threshold potential is reached, the neuron generates a new spike or a sequence 
of new spikes at the axon hillock that are transmitted via the axon to a postsynaptic 
neuron. This entire process is called Leaky Integrate-and-Fire (LIF). The term leaky 
reflects the fact that the cell membrane is not a perfect electrical insulator. Numer- 
ous LIF models, such as the FitzHugh-Nagumo, Morris—Lecar, or Hindmarsh—Rose 
models, have been developed to address different aspects of the biological sub- 
strate [158—161]. Depending on the electrical activity of two connected neurons, the 
connection strengths (the weights) can become weaker or stronger. This is at the 
heart of Donald E. Hebb’s learning rule, who first recognized that “Neurons which 
fire together wire together” [162]. On the biochemical level the variable strength is 
explained by the amount of neuro transmitters (vesicles) which are released into the 
synaptic cleft. 

From an engineering point of view, nervous systems process information in such 
a way that silicon technology, the holy grail of modern digital computing strategies, 
seems to be outmatched. For example, electronic components and circuits, such as 
transistors, memories and processors, are optimized for small parameter spreads to 
run at GHz clock frequencies under a precise pulse timing [47, 54]. In particular, 
they exploit nanosecond signal pulses that travel at nearly the speed of light along 
well-ordered transmission lines that connect different system parts in an essentially 
two-dimensional topology. In contrast, information pathways in nervous systems are 
characterized by highly irregular tissue consisting of neurons, synapses, and axons. 
Low conduction velocities on the order of several m/s lead to pronounced signal 
retardation, i.e. delays. In Fig. 5, characteristic timescales of CMOS processors and 
nervous systems are compared. In digital computing, the pulse duration is below a 
ns, and the signal transmission velocity is at nearly the speed of light. In nervous 
systems, the corresponding values are 3.5 ms for the pulse duration of an action 
potential or spike, and a few tenths of a ms for the transmission of a spike along 
myelinated axons [163]. Whereas the clock frequency of a modern Si processor is 
about 5 GHz, human EEG brain waves range from below | Hz to a few 100 Hz [41, 
164, 165]. This represents a six orders of magnitude discrepancy between technical 
and biological parameters. These facts alone point towards fundamental differences 
between information processing in digital computing and those in natural nervous 
systems. 


4.2 Global Aspects of Information Processing in Nervous 
Systems 


Nervous systems are considered to be time-varying networks in which spike- 
dynamics and cellular morphology are intricately linked and reciprocally interwoven 
[27, 151, 166, 167]. 

Information processing in nervous systems applies a broad range of structurally 
and temporally related phenomena [163, 168, 169]. At the level of individual 
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synapses, neurons, and axons, the formation and transmission of action potentials 
(“spikes”) are reasonably well understood. However, a look at the mesoscopic and 
macroscopic level of the three-dimensional neuronal network leads to an entirely dif- 
ferent assessment. Although groundbreaking progress has been reported on in vivo 
and in vitro techniques over the last decades, the nervous system’s spatiotemporal 
information processing is still not well understood [12, 170-172]. The biochemical 
mechanisms that explain higher brain functions at the cellular level, such as aware- 
ness, perception, and in particular consciousness, remain elusive [15, 24, 173-176]. 
Nonetheless, neuroscientists were able to identify basal mechanisms that define the 
fundamental platform of the unique and marvelous nervous system’s information 
processing. Characteristic features, such as STDP [141, 159, 177], stochastic fir- 
ing and bursting of neurons in the hundred Hz range, recurrent network structures, 
and aspects of oscillatory synchrony in larger neuronal ensembles [39, 41, 104, 
114, 178-183] are essential ingredients in biologically-based information process- 
ing. Moreover, factors related to the close interaction of a nervous system with its 
environment, i.e. external stimuli, are of crucial importance [184]. Therefore, neu- 
ronal design principles provide a model for bio-inspired computing systems, which 
are diametric to development strategies in present binary IT, including GHz clock 
frequencies, near-light-speed signal transmission, and clearly separated from logic 
and memory [47, 51]. 

Beyond that, we would like to emphasize that information-related aspects of ner- 
vous systems during evolution (phylogenies), along with their individual develop- 
ment throughout their lifetime (ontogenesis), provide a promising model from which 
novel electronic architectures may be designed. In the animal kingdom, the intricacy 
of nervous systems varies tremendously between single- and multi-cellular organ- 
isms, and the human brain with its billions of interconnected neurons [185-193]. For 
the sake of completeness, we would like to specify that the existence of cognitive 
functionalities in entities without a nervous system, such as plants or the acellular 
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slime mold Physarum polycephalum, is currently heavily debated. For interested 
readers, more detailed information can be found in the following Refs. [194-197]. 

Despite their different cognitive capabilities, neurons and nervous systems present 
many common features in all creature, such as synapses, signal transmission lines 
(axons), and action potentials (spikes), that act as basic information building blocks. 
While the term morphology defines the real structure of a nerve net, the topology 
of a net is more abstract, related to important theoretical graphical parameters that 
define the connectome of a nervous system [24, 27, 198, 199]. The connectome is 
considered to be the canonical state describing the cellular wiring diagram of a nerve 
net. Edges, nodes, cluster coefficients, characteristic path lengths, hubs, and motifs 
determine the topological quality of a net, for example. An unraveling of the micro- 
and macro-connectome and nervous system dynamics offer a suitable model for the 
next generation of bioinspired hardware electronics [169]. 

The network cube (Fig. 6a) classifies a number of different nets according to the- 
oretical attributes, including randomness, modularity, and heterogeneity [26, 200]. 
Interestingly, in this framework, cortical maps (lower right corner of the cube) 
extracted from the structural properties of nervous systems are somewhat isolated 
from all other nets, which are located in the upper left corner of the cube. In Fig. 6b, 
dynamical complexity (y-axis) is described as a state between complete asynchrony, 
with independent, random firing of the individual oscillators, and complete syn- 
chrony, with all oscillators firing in phase [201]. Between these two extremes, sys- 
tem dynamics can be characterized by a complex and time-varying interaction of 
the oscillatory ensemble. This regime exhibits features of self-organized criticality 
(SOC) typically observed at and near phase transitions and might be identified by 
avalanches of firing neuron ensembles [20, 23, 202-205]. Avalanche behavior is 
common in many physical phenomena, such as magnetic systems, earthquakes, and 
brain dynamics at the critical region of phase transitions, and were first described by 
Bak et al. [202]. The common feature of all these systems is slow external driving, 
causing an intermittent, widely distributed response. Avalanches appear in very dif- 
ferent sizes, often distributed in the form of power laws. As known from statistical 
physics, power laws imply the absence of a characteristic scale, a property observed 
close to a critical point. When describing the dynamics in a nervous system using the 
SOC and brain-like avalanches models, the type of phase transition associated to each 
term must be clearly defined. For example, SOC and brain-like avalanches in NWNs 
(see Sect. 3.2 and Fig. 3c) are related to non-activity - activity phase transitions. In 
the context of firing neuron ensembles in a brain, SOC and avalanches may describe 
a temporal phase transition between the asynchronous and synchronous states [203]. 
In other words, a system could be in the supercritical state (above the critical point) 
in an inactive -active phase transition, while remaining subcritical (below the criti- 
cal point) with respect to the asynchronous-synchronous phase transition. However, 
such phase transitions are not necessarily exclusive and might appear simultaneously 
in the brain, or the mechanisms could even be interwoven. So far, while nanopar- 
ticle networks and NWNs have been studied in context with their activity pattern, 
neuron-like oscillatory components have yet to be considered. The orchestra of firing 
neuron ensembles is considered a key underlining mechanism in understanding the 
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Fig. 6 Network cube and complexity: a classification of various networks. The dashed blue line 
illustrates a fictive, guided “walk” through the cube, starting from “S” and ending at the goal “G”. In 
this way, it will be possible to push the network properties of neuromorphic circuits towards those of 
cortical maps (see lower right-hand corner in the cube). This approach is part of an artificial spatio- 
temporal networks. b Qualitative illustration of the complexity term. From a spatial (topological) and 
temporal (dynamical) point-of-view, a complex system is neither completely random nor entirely 
ordered, but exhibits a state in between. a and b are adapted with permission from Solé and Valverde 
[200] and Huberman and Hogg [201], respectively 
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binding problem, i.e. the capability of the brain to integrate (bind) different sensory 
inputs. For example, such a process can occur in the visual system when forming a 
unified perception of the environment [37—40, 42, 206, 207]. Suggestions on ways 
to include relaxation-type oscillators to mimic the LIF features of biological neurons 
and the state of SOC are presented in chapter “Critical Discussion of Ex situ and In 
situ TEM Measurements on Memristive Devices”. Finally, the topological and tem- 
poral dynamics of the regime are extremely sensitive to external distortions (stimuli) 
at the critical point, allowing the system to respond in numerous ways to external 
stimuli [23, 25, 30, 208, 209]. In biological terms, this means that the system can 
easily adapt to risky environmentally-driven situations. The manifold brain states 
available near the point of criticality offer a wide repertoire of means to react in a 
reasonable way to external tasks imposed by the environment. In extreme situations, 
this improves the chances of survival and is of evolutionary importance. 


4.3 Phylogenies and Ontogenesis 


The origin of bio-inspired computing can be best drawn from the two following 
neuroscience quotations: 

(1) Gilles Laurent pointed out the common evolutionary heritage of living organ- 
isms. His contribution “Shall We Even Understand the Fly’s Brain? (see: 23 Problems 
in Systems Neuroscience edited by J. L. van Hemmen and T. J. Sejnowski, Chap. 1, 
p. 3, [210]) states: “When it comes to computation, integrative principles, or “cogni- 
tive” issues such as perception, however, most neuroscientists act as if King Cortex 
appeared one bright morning out of nowhere, leaving in the mud a zoo of robotic 
critters, prisoners of their flawed designs and obviously incapable of perception, 
feeling, pain, sleep, or emotions, to name but a few of their deficiencies.” [211]. 

(2) Martijn P. van den Heuvel et al., made in “The Neonatal Connectome Dur- 
ing Preterm Brain Development” the following statement: “The adult cerebral brain 
network is the result of a complex developmental trajectory. From the prenatal for- 
mation of the first neurons, throughout the first years of life and all the way into late 
adolescents, the brain undergoes an elaborate developmental trajectory.” [212]. 

How are these sayings so important for the design of novel bio-inspired com- 
puting primitives? The general idea behind these two quotations is the concept of 
development. Quotation (1) by Gilles Laurent highlights evolutionary development, 
the phylogenies of species, and their relevance to the emergence of the human cortex. 
This bottom-up approach favors the study of less complex creatures that appeared 
early during evolution, laying the foundation for much more complex nervous sys- 
tems in vertebrates [190]. In particular, information processing strategies throughout 
evolution and in completely different species are astonishingly similar, if not the 
exact same. For example, the basic ingredients of information processing (neurons, 
synapses, and action potentials, as described in chapter “Redox-Based Bi-Layer 
Metal Oxide Memristive Devices”) in the nervous systems of squids and macaques 
are hardly distinguishable from one another. Although François Jacob addressed 
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the random and playful character of evolution by the phrase “Nature is a tinkerer, 
not an inventor” [153], evolution can be somewhat conservative in the sense that 
similar structural and dynamical features appear in very different species through- 
out phylogenies. This justifies the investigation of information pathways in simpler, 
easier-to-understand organisms in order to comprehend higher brain functions in 
more complex vertebrates. A famous example is the research of Eric Kandel on 
the snail Aplysia, relating physiological signaling with behavior [185]. Studying the 
neural design of biological species with only a few hundred or thousands neurons is 
a fruitful ansatz to develop novel computing primitives [169, 191-193, 213-215]. 
We will come back to this issue in chapter “Critical Discussion of Ex situ and In situ 
TEM Measurements on Memristive Devices”. 

While phylogenetics addresses the development and evolution of groups of sim- 
ilar species, ontogenesis is the study of how an individual member of a species 
develops as it ages. In quotation (2), Martijn P. van den Heuvel and coworkers under- 
line the intriguing mechanisms of nervous systems development in humans, from 
conception to late adolescence. We propose that ontogenesis and their functional 
ingredients could serve as an essential guideline for novel computing primitives. 
To support this argument, we describe here the fundamentals of ontogenesis in the 
human nervous system, including the importance of external stimuli during devel- 
opment. Physiology, neurobiology, and behavioral science provide overwhelming 
experimental evidence showing that the conditions during the growth and regenera- 
tion of neuronal nervous systems under external stimuli are of central importance [64, 
216-221]. Both the formation and elimination of nerve cells, synapses, and axonal 
connections occur frequently during the first stages of brain development, belonging 
to a very creative process that shapes the nervous system to be well-adapted for future 
environment-related tasks. In addition to the creation of neurons and axons, pruning 
(programmed cell death or apoptosis) and axonal rewiring are both essential and 
expedient mechanisms. Finally, myelination of axons is an essential step to improve 
the nervous system’s performance by shaping and optimizing the signal transfer time 
between neurons and distributed brain areas. Gerald Edelmann coined the expres- 
sion “Neuronal Darwinism” to highlight the striking parallels between evolution 
and brain development (Edelman 1987; Tononi, Sporns, and Edelman 1994; Edel- 
man and Tononi 2001; Van Ooyen and Butz-Ostendorf 2017) [222-225]. Neurons, 
synapses, and axonal connections grow lavishly at first, a growth that is controlled 
by the genome, epi-genome, and stochastic factors. Subsequent structural shaping 
and elimination, often called blooming and pruning, are largely influenced by the 
interaction of the entire nervous system with environmental stimuli, and the ner- 
vous system’s subsequent reaction [184, 226, 227]. There have been attempts in 
the past to design materials and systems that mimic biological information process- 
ing, dubbed “evolvable hardware” and “evolution-in-materio” [97, 196, 197, 228, 
229]. This work has been recently extended to novel, transistor-based devices by 
Baek et al. [230]. Although the findings are very promising, basal spatiotemporal 
and topologically-relevant mechanisms have not been reproducible in electronics 
hardware so far. In both biological and artificial systems, the connection between 
these mechanisms should be worked out with regard to the required complexity and 
functionality (see Fig. 6). 
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Fig.7 Structural plasticity. In vivo time snapshots of the appearance and disappearance of dendritic 
spines in the mice barrel cortex. The top row shows the growth of a persistent spine between days 12 
and 28 (orange arrows). The bottom row shows examples of spine retraction (yellow arrows between 
days 0 and 16, and green arrows between days 0 and 12). Figure from [241] with permission. These 
structural changes were correlated to external stimuli applied by whisker trimming in mice 


Neural network growth in nervous systems has been studied in-depth both the- 
oretically and experimentally [29, 31, 32, 212, 219, 231-234]. In particular, the 
early stages of nervous system growth under external stimuli is of critical impor- 
tance for the healthy development of mature creatures [157, 220, 235]. It is known 
that both external stimuli and genetic factors have tremendous impact on the emer- 
gence of functional neural circuits that determine behavior during critical periods 
of cortical region growth [226, 227, 234, 236]. Cell overproduction and subsequent 
attrition are likewise important for nervous system development [226, 237, 238]. 
Morphological aspects, connectivity, growth, regeneration, and the impact of neu- 
ronal activity-related spike-based synchronization mechanisms in neuronal network 
ensembles serve as models for novel electronics [39, 41, 114, 165, 175, 239, 240]. 
Clear evidence of structural dendritic spine plasticity is shown in a series of pho- 
tographs taken over a few days in Fig. 7, demonstrating that spines grow and shrink 
depending on external, touch-related stimuli in mice [241]. 

A look at a few growth parameters underlines the importance of understanding 
biological networks during development. A two-year-old human toddler exhibits the 
maximum number of neurons and synapses of our species, roughly a factor of two 
more than a fully grown adult. If we estimate 170 billion neurons [187, 226] with 
10° synapses per neuron, a two-year-old human carries 170 x 10!* synapses. We 
assume the total axon length of a toddler to be about 850,000 km (https://aiimpacts. 
org/transmitting- fibers-in-the-brain-total-length-and-distribution-of-lengths/). The 
time between egg fertilization to the age of two is 1000 d or 8,64 x 10’ s. This 
leads to an average net growth of roughly 2000 neuron/s, 2 million synaptic inter- 
connections/s and an axon growth rate of about 10 m/s! These measures alone unam- 
biguously demonstrate the overwhelming significance of network growth in humans, 
particular during childhood [221, 242]. Moreover, we believe that such a tremendous 
development is an interesting template for novel computing architectures. It might 
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Fig. 8 Blooming and pruning of nerve cells in young humans [157] and J. Conel, “The Post-Natal 
Development of the Human Cerebral Cortex,” Harvard University Press, Cambridge, 1939-1967. 
Adapted from [157] 


be an essential building block to achieve higher brain functionalities in artificial 
systems, and constitutes a key aspect artificial spatio-temporal networks. 

Figure 8 shows several snapshots taken during human development, where the 
excessive growth of neurons between the ages of one month to two years is clearly 
visible. Interestingly, between the ages of two to four years, neuron pruning leads 
to reduced neuron density. While the net neuron density during adulthood is rather 
constant, blooming and pruning still continue to occur, albeit at a much lower rate 
[225]: 

From the postnatal phase up to the age of around two years, our central nervous 
system is characterized by enormous development and permanent remodeling, while 
being simultaneously subject to an exuberant amount of external stimuli via our 
senses [184, 212, 227, 232]. Genetics, stochastics, and external stimuli (in other 
words nature and nurture) define who we are and strongly influence higher brain 
function during adulthood, including perception, awareness, and consciousness. 

In Fig.9, windows of plasticity in human brain development are sketched [227, 
243]. Even in much simpler creatures (e.g. the worm C.-elegans), external stimuli 
play an essential role in the healthy development of the nervous system [235]. 

These windows for sensing, motor skills, and higher cognition are also called crit- 
ical periods. They reflect the tremendous rearrangement of the human brain during 
early childhood, accompanied with enormous learning capabilities. It is interesting 
to assign the above estimated growth parameters and the appearance of critical peri- 
ods to human altriciality. Altriciality refers to the way creatures are born completely 
incapable of caring for themselves (Dunsworth et al. 2012). Hence, at the moment 
of birth (eye opening), a sudden rush of external stimuli, in particular vision, meets a 
premature nervous system still under heavy construction, reconstruction, and growth, 
in the case of humans. The concomitant occurrence of environmental input, nervous 
system growth, and close interaction between the nervous system and its environ- 
ment may explain the huge plasticity and learning capabilities during these first 
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Fig. 9 Illustration of critical or sensitive periods during the first years after birth for humans. The 
three periods present (from left to right) the development of sensory pathways, motor skills, and 
higher cognitive functions. Adapted from [227, 243] with permission 


years. This development seems to be essential to form higher brain functions [212, 
218]. Although it might be incredibly difficult to mimic such basal neurobiological 
mechanisms in engineered systems, nervous system development and growth cannot 
be neglected in establishing higher brain functions in artificial systems. Attempts to 
achieve this goal are proposed in chapter “Critical Discussion of Ex situ and In situ 
TEM Measurements on Memristive Devices”. 


4.4 Homeostasis 


As in Sect. 4.3, we begin with the following sequentially-labelled neuroscience quo- 
tation: (3) Arjen van Ooyen and Markus Butz-Ostendorf emphasized the role of 
homeostasis on p.133 of their contribution (see: The Functional Role of Critical 
Dynamics edited by Nergis Tomen, J. Michael Herrmann, and Udo Ernst [244]: “In 
conclusion, during development, homeostatic structural plasticity can guide the for- 
mation of synaptic connections to create a critical network that has optimal functional 
properties for information processing in adulthood.” [245]. 

Roughly speaking, is homeostasis a kind of counteracting mechanism to network 
plasticity, and thus an important factor to ensure network robustness and stability. 
As will be discussed below in more detail, homeostasis comprises dynamical and 
morphological components, and is thought to explain how a nervous system stabilizes 
(itself) near the point of criticality [246]. In other words, homeostasis addresses the 
term “self” in SOC. The role of homeostasis as a stabilizing factor in neural networks 
is amply described in a huge number of publications, with only a few mentioned here 
(Abbott 2003; Turrigiano 2012; C. Tetzlaff et al. 2010; Stepp, Plenz, and Srinivasa 
2015; Fauth, Worgotter, and Tetzlaff 2017; van Ooyen 2017; Ma et al. 2019) [225, 
239, 247-251]. In homeostatic structural plasticity, all incoming synapses of a cell 
are modified to stabilize the neuronal activity around a particular level (set point), 
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and reflect a negative feedback loop between neuronal activity and connectivity [225, 
248, 252, 253]. The fundamental principle of homeostasis is sketched in Fig. 10. 

Higher firing (dynamic component) of a neuron results in spine deletion (morpho- 
logical component), whereas reduced firing supports spine formation, keeping the 
average electrical activity at a set-point, potentially stabilizing the global activity of 
the entire neural ensemble near the desired critical state, i.e. the state with the largest 
dynamic range for information processing [30, 208, 209, 254]. While this model 
appears attractive at first glance, it raises a fundamental question in neuroscience: 
“How can an individual, local neuron in a huge nervous system access the global 
network state in order to orientate its own activity accordingly?” [27, 203], or in 
other words, what defines the activity set-point? This is an example of the poorly 
understood relation between local, mesoscopic, and global mechanisms in nervous 
systems. 

In chapters “Redox-Based Bi-Layer Metal Oxide Memristive Devices” and 
“MemFlash—Floating Gate Transistors as Memristors’”, we presented various basal 
local and global information pathways in nervous systems. In the following chapter, 
we will suggest a number of strategies with the goal of implementing higher brain 
functions in artificial systems [229]. 


5 Artificial Spatio-temporal Networks 


At this point, an obvious and understandable question might be: Is the goal to achieve 
higher brain functions in artificial systems possible at all or, more precisely, to what 
extent can the intriguing and complex biological mechanisms described in chap- 
ters “Redox-Based Bi-Layer Metal Oxide Memristive Devices” and “MemFlash— 
Floating Gate Transistors as Memristors” be merged into a novel computing prim- 
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itive? How close is neuroscience to understanding higher brain functions and to 
what extent can the plethora of phenomena set by materials and engineering designs 
strategies enable mental functions in artificial systems? 

Here we discuss possible ways and limitations of using artificial systems to mimic 
biological fundamentals, including topological and dynamical aspects, such as phy- 
logenies, ontogenies, homeostasis, SOC, memory, oscillatory orchestra (synchrony), 
and so on. Nonetheless, we are aware that fundamental limits which may impede 
consciousness in engineered systems. It would be interesting, however, to identify 
and define those limits. 

In Fig. 11, considerations set by materials science and design strategies are 
illustrated. 

Before describing the interplay between the components sketched in Fig. 11, we 
should first consider a few aspects of biological information pathways which are 
obviously implementable by materials science and electronics, and might simplify 
the execution of the proposed artificial spatio-temporal network. In a human brain, 
the ability to access, and thus measure, the structural, topological, and dynamical 
states is hindered by both technological and ethical constraints [255, 256]. In contrast, 
artificial systems should theoretically permit access to all local and global parameters 
in any conceivable experimental setup. This offers a high degree of freedom in 
designing artificial systems. In particular, for a system growing in complexity, a 
designer might decide which segments should be externally controlled and which 
should develop via self-assembly and self-organization. 

Furthermore, the time scales involved in biological information processing may 
actually facilitate their artificial engineering. In phylogenetic and ontogenetic devel- 
opment, low time scales dominate the scene. Species vary from one generation to 
the next, with networks growing from days to years. Additionally, nervous system 
dynamics are in the 100 Hz range, with low transmission velocities on the order of 
m/s, i.e. the speed of spikes along axons are common. As such, there is no need to 
build ultrafast artificial systems in order to imitate basal biological information path- 
way. Indeed, the deposition or synthesis of any material, e.g. nanoparticles or NWNs, 
is a growing materials network (Fig. 1 1a), and can be adjusted to low time scales. In 
addition, low time scales adapt well in many ways to materials transport parameters, 
including ionic drift, diffusion currents, and mass transport in general. Biological 
time scales are easily accessible by electronics, facilitating circuit design, and per- 
mitting real-time observation of spatio-temporal system development (Fig. 1 1b, c, d). 
For example, leaky-integrated-firing of a biological neuron can be technically real- 
ized by van der Pol (vdP) oscillators [115, 119, 257], compact devices based on VO2 
or NbO,, which exhibit a negative differential resistance (NDR) I-V curve [126, 258— 
260], or integrated, mixed-signal circuits [120]. In general, low time scales known 
form biological information pathways, including external stimuli that affect them, 
offer an exploration space attainable by materials-related phenomena, electronics, 
and parameter monitoring. 

How can an artificial spatio-temporal computing system, as sketched in Fig. 11, 
be practically realized? The goal in a bio-inspired artificial spatiotemporal network 
is to reach the desired topological and dynamical states simultaneously, in order to 
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Fig. 11 Artificial spatio-temporal networks: Materials science considerations and design strate- 
gies to generate higher brain function in artificial systems. The proposed system takes basal func- 
tionalities of bio-inspired information pathways into account discussed in chapters “Redox-Based 
Bi-Layer Metal Oxide Memristive Devices” and “MemFlash—Floating Gate Transistors as Mem- 
ristors”. a 2D or 3D spatio-temporal materials network. Wires within the network are connected via 
memristive components. The memristive functionality at cross-sections of the network implements 
memory and local plasticity in the network. The faded area represents a growing network. In the 
case of a 2D network structure formed on a planar substrate, network growth might be modified, for 
example, by pre-pattern substrates, a functionalized surface, additional electrical potentials, optical 
stimuli, and deposition-related growth parameters (materials, deposition rate, reactive gases, sub- 
strate temperature, and so on). A 3D network allows further freedom of design and allows for a 
nervous system-like connectivity. The 3D network could be in a solid phase, or even multiphase, 
network, the latter combining materials in the solid, liquid, and gas phase. b Representation of a 
pulse-oscillator ensemble in order to mimic neural spiking activity. The individual oscillators of 
the ensemble are electrically connected to the network, leading to modifications of the network 
connectivity by oscillator pulses. Conversely, the network weights in turn influence the dynamic 
state of the oscillator ensemble via pulse coupling. The oscillatory ensemble allows an input of 
external stimuli (e.g., touch, vision, and hearing) via fire rate coding. In addition, analyzing the 
interspike interval (ISI) distributions of the ensemble in quasi-real-time enables permanent mon- 
itoring of the dynamic state of individual oscillators, as well as the entire ensemble. ¢ Stage to 
monitor the structure and extract the topology of the network in real time by, for example, optical 
microscopy, electron microscopy, thermal imaging, or magnetic field distribution detection (similar 
to MEG (magnetoencephalography)). d By monitoring the oscillatory ensemble dynamics (see b) 
and the structural connectivity (see c), the spatio-temporal state and its evolution can be analyzed 
in real-time 
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mimic the previously discussed characteristic hallmarks of the nervous system. This 
is handily illustrated in both Fig.6a, where the topological cortical map region is 
labeled “G” (Goal) in the network cube, and in Fig.6b, where the state of SOC is 
highlighted as the envisaged dynamical state. The main challenge here is to define the 
appropriate material network properties and dynamical setting for the entire system 
that will enable a similar spatiotemporal state to that of a nervous system. This global 
system state is often said to be structurally complex while being temporally close to 
the edge of chaos [24, 25, 171, 173, 183, 261, 262]. To achieve this goal, we describe 
the components presented schematically in Fig. 11 and their interactions in accor- 
dance to the biological information pathways described in chapters “Redox-Based 
Bi-Layer Metal Oxide Memristive Devices” and “MemFlash—Floating Gate Tran- 
sistors as Memristors”. The material network template offers manifold opportunities 
on either a 2D or 3D platform (Fig. 11a). A network growth mimicking ontogenesis 
can be realized by continuous film deposition, or ongoing material synthesis of, for 
example, nanoparticle or nanowire networks. Network growth can be influenced in at 
least in three ways, the first of which being the oscillatory ensemble that is electrically 
connected to the network. Here, external stimuli, e.g., hearing, vision, and touch, are 
imprinted into the material network growth process via fire rate coding (Fig. 1 1b). 
Network formation and structure evolution are modified by the additional potential 
differences between the oscillator contacts within the network. Second, by integrat- 
ing additional conductive pads (islands) on a 2D substrate platform, the formation of 
filaments between the oscillator’s electrodes can be controlled by the islands’ shape, 
number, size, and/or additional applied bias potential (Fig. 11a). The formation of 
conductive filaments during network growth could also be manipulated via struc- 
turally modulated or functionalized surfaces. In this way, not all network pathways 
are allowed, while others are assisted [263]. Biologically, this corresponds to axon 
growth and guidance [233]. This approach can also apply to 3D structures, which 
provides an increased degree of freedom and in principle allows nervous system-like 
connectivity. The materials network, whether 2D or 3D, does not necessarily have 
to be in the solid state: Electrolytes may be an appropriate fluid which satisfies the 
aforementioned requirements, including the state of criticality [21, 99, 264-269]. 
Third, additional stimuli (Fig. 1 1a) in the form of, e.g. light or temperature, can also 
modify the spatio-temporal evolution of the functional material network. An imprint 
of information during network growth is common to all three methods. This distin- 
guishes the artificial spatio-temporal network approach from common AI systems. In 
the latter, the training or learning sequence is applied after system manufacture. By 
applying the three methods described above, it might be possible to imprint informa- 
tion in a similar way to that of a human nervous system during ontogenesis (method 
1), as well a kind of a-priori knowledge (methods 2 and 3), i.e. phylogenetic factors. 

For the entire system, a simultaneous, in-depth monitoring of network structure 
during its development and temporal evolution is intended, in accordance with a 
neuroscience approach to extract the structure and dynamics of complex brain net- 
works [24, 270]. To this end, the spatio-temporal development of time-varying con- 
nectivity within the functional materials network (see Fig. 11a) will be monitored, 
for example, by means by optical microscopy, electron microscopy, or the magnetic 
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field distribution in accordance with magnetoencephalography (MEG) (see Fig. 11c). 
This will allow visualization of the time-evolving correlation matrix of the oscillatory 
ensemble, and the extraction of more theoretical metrics, such as cluster coefficients, 
characteristic path lengths, motifs, modularity, and hubs [24, 26, 27, 171]. In Fig. 6a, 
the pale blue dashed line in the cube represents a fictional pathway through the net- 
work cube. At first, we assume that the materials network is a topology state labeled 
“S” (Start). The position “S” within the cube is chosen as an example, but could just 
as well be any other topological position within the network cube. By constantly 
monitoring the topology of the system during network growth and intervention via 
a set of parameters (e.g. added materials, extra potentials, and external stimuli to 
the oscillatory ensemble), it might be possible to adjust the system to arrive at “G”, 
defined by a set of characteristic theoretical parameter (hubs, motifs, modularity, 
cluster coefficient, path length, etc.) [27]. 

Simultaneously, the ISI distribution and time series of the oscillatory ensemble 
will be recorded (see Fig. 1 1b) [271]. Spike train distances provide a means of quan- 
tifying neuronal variability and the degree of synchrony in and between oscillatory 
ensembles, and may indicate the rise of oscillatory avalanche firing as one indicator of 
the SOC [23, 272-277]. SOC is described as a state located somewhere between the 
random, independent firing of individual oscillators, and complete synchrony, where 
all oscillators fire in phase with the same frequency [265, 278]. Between these two 
extremes, a system’s dynamics can be characterized by a complex and time-varying 
interaction of the oscillatory ensemble (see Fig. 6b). This regime exhibits features 
of criticality typically observed close to phase transitions [20, 25, 103, 209, 279- 
281]. In particular, the topology and temporal dynamics of a system in such a state 
are extremely sensitive to external distortions (stimuli) and may respond to them in 
numerous ways. 

Practically speaking, we will begin by analyzing coupled nonlinear oscillator 
network raster plots, phase portraits, phase response curves, bifurcation diagrams, 
spike distance measurements, and cross-correlation type time-series analysis. Infor- 
mation from these analyses will be subsequently applied to quantify the phase and 
frequency relationships between network oscillators and their development over time 
[282, 283]. Finally, we would like to discuss obvious obstacles and challenges. In 
Sect. 4.4, the rule of homeostasis was highlighted. The concept of homeostasis is of 
essential importance to stabilize the nervous system dynamics and morphology to a 
set-point. For the system presented in Fig. 11, homeostasis is not illustrated. It might 
be possible to reconstruct a feedback parameter from the structural and functional 
matrices to reduce or enhance, if necessary, the oscillatory activity, or to modify the 
material growth process. Another challenge might be the implementation of appro- 
priate delay lines to mimic the important signal retardation known from nervous 
systems [173, 182]. Ionic conductors with slow ionic motion in the form of drift or 
diffusion currents could be a possible solution. 

One important issue remains: Picture a fabricated artificial spatio-temporal system 
as depicted in Fig. 11, that presents all previously discussed biological information 
pathways. How can we benchmark the system, and determine how it solves tasks 
set by external stimuli? Certainly, the functional and structural network states reflect 
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the overall system state. As such, one viable approach is to read out the system state 
and to activate a set of artificial motor neurons to react to an input task. However, 
this does not accurately represent the process in the human brain, where there is no 
internal, global system observer to decide on the next step [9]. At this point, we are 
confronted with a difficult challenge: how can we lead matter to imitate the mind? 
While the authors can suggest an example system as shown in Fig. 11, this question 
remains open. 


6 Benchmarking for Bio-inspired Computing 


Benchmarking in AI is an important approach to measure its performance, and sub- 
sequently enable comparisons between different systems. In pattern recognition, for 
example, MNIS data sets are used, with the recognition rate defining a clear bench- 
mark. While contemporary AI systems show extraordinary capability in performing 
a single, specific task, their success at task variability is highly limited compared to 
the nervous system. Nonetheless, a new generation of AI has demonstrated extraor- 
dinary capabilities in the field of gaming (Chess and Go), including an aptitude for 
self-learning [284]. Yet, it remains unclear how to define a fair and comprehensible 
benchmarking for neuromorphic systems and bio-inspired computing [285]. Com- 
putational tasks must be carefully designed in order to assess the overall system’s 
performance in comparison with human mental capabilities, as previously proposed 
by Alan Turing in his seminal work on Machinery and Intelligence [155]. Bloom’s 
learning taxonomy, which was developed to hierarchically categorize learning in the 
classroom, can be helpful in assessing how successfully artificial systems mimic 
higher brain functions [286]. This taxonomy contains six categories of cognitive 
skills and presents a hierarchy with increasing cognitive functionality from bottom 
(factual knowledge) to top (creation) (see Fig. 12), or in other words, from lower- 
order skills that require less cognitive processing to higher-order skills that require 
deeper learning and a greater degree of cognitive processing [287]. 

This strategy may serve as a basis for benchmarking in bio-inspired computing 
systems. However, due to the nervous system’s task variability for each of the six cog- 
nitive categories, transparent benchmarks must be developed. This goal is extremely 
important for future comparisons of bio-inspired systems, which are currently devel- 
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oped on different platforms. In addition, resource-related parameters, such as energy 
consumption, system weight, and failure tolerance, need to be included. 


7 Discussion 


This perspective introduces the concept of artificial spatio-temporal networks, which 
proposes basal hallmarks, such as morphological and dynamical characteristics of 
nervous systems, to reproduce higher brain functions in artificial systems. In par- 
ticular, the basal mechanisms known from the growth of nervous systems might 
play a significant role in their function. This concept will undoubtedly be a way to 
include biologically-relevant features in future artificial systems. Yet, only the tip of 
the iceberg has thus far been addressed: to fully realize an artificial spatio-temporal 
network, several challenges remain unresolved. 

In more general terms, artificial spatio-temporal networks again raise the fun- 
damental question: “To what extent can higher brain functions be reproduced in 
artificial systems?” Seminal books and papers by [1, 3, 5, 7, 176, 234, 288], and 
many more address this topic in one way or another. According to the authors, higher 
brain function can be described on the basis of natural sciences and mathematics, 
permitting us to view this challenge in another light. On an atomistic level, we find in 
living nature, and therefore in any nervous system, old and well known friends from 
the periodic table of the elements, including but not limited to Carbon (C), Sodium 
(Na), Potassium (K), Chlorine (Cl), Oxygen (O), and Hydrogen (H). Any effort to 
establish higher brain function in an artificial system, whether in silico (software 
oriented) or in a material-based substrate, as in the case of artificial spatio-temporal 
networks, should apply another tool box of elements to establish awareness, percep- 
tion, or consciousness, e.g. Silicon(Si), Gold (Au), Silver (Ag), Tungsten (W), O, and 
so on. There is no obvious reason why this strategy should not work, but if it cannot, 
what are the fundamental limits, and how are they defined? A look at biochemi- 
cal substrates in living species highlights the weaknesses of the simplistic, atomistic 
view point. There is still unknown genetic information that strongly controls nervous 
system behavior and function, especially during development, which therefore can- 
not currently be considered in any artificially-constructed systems. Whether there are 
shortcuts to bypass the role genes play in neural behavior and development is com- 
pletely unknown, and might act as a show stopper [234]. On the one hand, it is truly 
challenging to introduce basal biological functionalities, such as homeostasis, signal 
delay, growth, and the appropriate states of criticality and topology in an artificial 
system. On the other hand, the materials tool box may offer plethora of phenomena 
which have not yet been explored for novel computing architectures [66, 266-269]. 
Hence, these simple questions and views point towards an even more fundamental 
aspect: in living systems, the separation between matter and information becomes 
blurred, making it risky to apply these terms without investigating living and artificial 
systems equally, or precisely clarifying the respective context [289]. 
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8 Conclusion 


In this perspective, we addressed fundamental limits of current ICT and briefly sum- 
marized the state-of-the-art. Today’s digital electronics work with clock rates in the 
GHz range, utilizing ns pulses and signal transmissions at nearly light speed in a 
vacuum. Meanwhile, nervous systems exhibit numerous remarkable and fascinat- 
ing features, including anticipation, awareness, perception, and consciousness. The 
associated action potential spikes are 6 orders of magnitude longer, and travel with a 
velocity 6 orders of magnitude lower, than their electronic analogs, while dissipating 
only a couple of Watts of power. We touched on the fundamentals of information 
processing in biological (nervous) and engineered systems. Specifically, we high- 
lighted the dynamical and morphological properties exhibited by nervous systems 
using the human brain as an example. The exceptional topology of the human cortex 
in comparison to other biological and technical networks, in addition to the state of 
SOC, served as guidelines to develop artificial spatio-temporal systems. A pathway 
to realize artificial spatio-temporal systems in a hardware-orientated system was pre- 
sented, aiming to emulate higher brain functions in an artificial system. The role of 
ontogenesis was discussed, revealing that the mechanism of neural network growth 
provides crucial information useful in designing novel artificial computing systems, 
which have yet to be addressed in great detail. 

Neural network growth illustrates how important the ongoing interaction between 
the internal and external world is when artificially creating the basic structures that 
provide the ability to learn specific functions. In our opinion, this emphasizes the 
importance of basal properties which, while beginning to be applied systems in 
artificial, have yet to be fully implemented. These properties include individual 
autonomous dynamic units, time-variable coupling between them, and both posi- 
tive and negative connection growth. With respect to time-variability, the research 
field has shown enormous progress in recent years with the development of mem- 
ristive systems. Although memristive devices can already replicate the phenomena 
associated with learning to a certain degree, the question remains whether these 
devices can suitably reproduce both the necessary processes in their entirety, and 
global dynamics which are shaped by an overwhelmingly complex network. The last 
point in particular presents immense challenges for a conservative implementation 
of memristive devices in large-scale systems. Finally, we discussed possible limita- 
tions in implementing higher brain functions in artificial systems. We concluded that 
genetic information plays a key role in the development of neural nervous systems, 
knowledge that we are still lacking if we want to fully implement this behavior in 
artificial systems, specifically with regards to awareness, perception, and conscious- 
ness. The exploration space for implementation is certainly extraordinary large for 
artificial spatio-temporal systems. This huge parameter space is both curse and bless- 
ing: while such a large number of variables must be monitored and controlled, it also 
allows for greater flexibility and opportunities. One thing is certain in this context: no 
matter which engineered solution ultimately prevails, humanity will be confronted 
with a multitude of ambivalent questions and challenges, in which certainly “Matter 
and Mind Matter”. 
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Abstract This chapter addresses opportunities for memristive devices in the frame- 
work of neuromorphic computing. Memristive devices are two-terminal circuit ele- 
ments, comprising resistance and memory functionality. This simple and likewise 
ingeniously concept allows beneficial applications in numerous neuromorphic cir- 
cuits. However, the electrical characteristics as well as the materials and techno- 
logical framework of memristive devices need an optimization for each specific 
application. The chapter starts with a short overview of basic principles of biological 
data processing followed by a taxonomy of different bio-inspired computing archi- 
tectures, divided into time-dependent and time-independent concepts. Furthermore, 
the requirements on particular memristive device properties, such as Z-V linearity, 
switching time, retention, number of states, time-dependency, and device variability, 
are discussed. The results of tangible examples of digital and analog memristive 
switching devices used in a deep neural network based on CMOS-integrated resis- 
tive random access memory devices (RRAMs) for chronic obstructive pulmonary 
disease (COPD) detection, in stochastic learning, in bio-inspired analog learning, 
and, finally, in oscillatory computing are presented and discussed. 
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1 Introduction 


While traditional von Neumann computing (binary and serial) continues to dominate 
the information and communication technology (ICT), recent groundbreaking inno- 
vations in alternative computing architectures and advanced electronic devices have 
become hard to ignore [1—6]. The mature silicon (Si)-based, complementary metal 
oxide semiconductor (CMOS) technology behind von Neumann computing led to 
tremendous progress in digital computing power over the last six decades. Moore’s 
law, the prediction that the device integration density on a chip doubles every two to 
three years, and accompanied by scaling laws, served as secure guidelines for CMOS 
engineers [7, 8]. Today’s Si-CMOS technology is characterized by impressive tech- 
nical specifications. To mention but a view, the smallest feature size in advanced 
Si-CMOS transistor is in the order of 10 nm, several billion functional transistors are 
integrated into a single arithmetic logic unit (ALU), and a von Neumann computer 
is running at clock-frequencies of about 5 GHz [5]. Nonetheless, this success should 
not disguise the fact that Si-based von Neumann computing reaches its limits in the 
near future, and system performance progress has slowed down for a couple of years. 
For example, although a further shrinking of transistor dimension is still possible, 
limitations, such as quantum mechanical tunneling, will set fundamental constraints. 
The so-called memory gap appeared as another architecture-related show stopper. 
While the clock frequency increased with every new chip generation, the memory 
access time did not follow this trend. Therefore, the system performance is limited 
because the data handling between the ALU and memory presents a data bottleneck 
[9]. Aware of this stagnation, worldwide efforts by universities, research institu- 
tions, and industry focus on numerous novel computing architectures and advanced 
functional devices. Besides quantum computing, bio-inspired computing attracted 
considerable attention [10-15]. The term “bio-inspired” embraces various classes of 
computing architectures and hardware systems that take, to a more or less extent, 
biological information pathways as guidelines into account. In order to categorize 
such bio-inspired computing systems, basic principles of information pathways in 
nervous systems are summarized here. We want to emphasize, that the following 
introduction to the biological background describes the information processing in 
nervous systems in a very simplified way and represents only the tip of the iceberg. 
For more details on the intriguing biochemical and spatio-temporal mechanisms in 
this context, we refer the reader to the literature [16, 17]. A common way to explain 
information pathways in nervous systems is to distinguish the processing on the local 
neuron level from that of the entire system. Neurons are the information processing 
units in nervous systems. Neurons receive and process information in the form of 
action potentials. In Fig. la, the structure of a neuron, including the soma, dendrites, 
the axon, and connections to other neurons by synapses, is sketched [16]. An action 
potential (spike) is an abrupt transitory and transmitted change of the resting poten- 
tial across the cell membrane. The amplitude of a spike is 100 mV, and its pulse 
duration is about 3.5 ms. Post-synaptic neurons receive action potentials (signals) 
from pre-synaptic neurons via dendrites and synapses. Those action potentials are 
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Fig. 1 a Blueprint of a neuron, including an enlarged sketch of a synapse and the illustration 
of a single action potential (i.e., a spike). Reprinted with permission of the corresponding author 
from [18]. b Change of synaptic weight (AW) in dependency on the timing (At = tpost — tpre) 
of pre-synaptic spikes at time tp,-e and post-synaptic spikes at time tpost illustrating spike-timing 
dependent plasticity (STDP). A W is measured 20-30 min after inducing the synaptic change with 60 
correlated pre- and postsynaptic spikes with a frequency of 1 Hz at synapses between hippocampal 
glutamatergic neurons in culture. The data is fitted with exponential functions of the form AW = 
e(—4!/7) for At > 0 (LTP) and At < 0 (LTD). Adapted from [19] [G.-q. Bi & M.-m. Poo, Annu. 
Rev. Neurosci. 24, 139—166 (2001)], with the permission of Annual Reviews. c Different irregular 
signal patterns, which occur in nervous systems [20]. ©2003 IEEE. Adapted, with permission, 
from [E. Izhikevich, IEEE Transactions on Neural Networks 14, 1569-1572 (2003)] 
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integrated within the post neuron’s cell body. If the potential exceeds a threshold at 
the axon hillock, a new or a series of new spikes are generated at the axon hillock. 
Such a spike or spike trains are transmitted via the axon to subsequent neurons. The 
process is called leaky-integrate-and-fire (LIF), reflecting that the cell membrane is 
not a perfect insulator. The conduction velocity along the axon is between a few 
cm/s up to a few tenth m/s. We like to underline that the spike duration (3.5 ms) and 
the conduction velocity (e.g., 100 m/s) are six orders of magnitudes smaller than 
the pulse duration (1 ns) and signal transmission speed (roughly the speed of light 
in vacuum, 3 - 10° m/s) in modern processors, respectively. Facts, which indicate a 
different information processing of von Neumann computers and nervous systems. 
The mentioned interconnects are called synapses. Interestingly, the synaptic effica- 
cies are not fixed and can change, which refers to the information coupling strength 
between the pre- and post-neuron. The efficacy can be increased (i.e., synaptic poten- 
tiation) or decreased (i.e., synaptic depression). This plastic behavior can last from 
milliseconds to minutes [called short-term potentiation (STP) or short-term depres- 
sion (STD)] or from hours to years and up to the whole lifetime of animals [called 
long-term potentiation (LTP) or long-term depression (LTD)] [17]. The change of 
synaptic coupling strength depends on the common activity of the pre- and post- 
synaptic cells, e.g., in accordance to signal timing between the pre- and post-neuron. 
This mechanism is called spike-timing-dependet plasticity (STDP) [21]. STDP is 
closely related to Hebb’s learning rule, which says that neurons that fire together 
wire together [22]. This rule describes the learning process in neural networks on 
the local cellular level by adjusting the synaptic efficacy dependent on the common 
activity of pre- and post-neurons. This further contributes to associative learning on 
the network level [23, 24]. Accordingly, STDP represents a local learning rule and 
is an essential process for establishing learning and memory in nervous systems [see 
Fig. 1b][19, 25]. In stark contrast to von Neumann computers, the spike generation 
in nervous systems is far from being regular. In dependency on the signal input of a 
neuron, a plethora of different firing rates (ranging from Poisson-like to bursting) are 
observed [26-28]. In Fig. 1c, examples of such irregular signal patterns are sketched 
[20]. Any signal transmission in nervous systems is accompanied by errors and noise 
[29]. For example, random potential fluctuations and the granular structure of neu- 
rotransmitters lead to a pronounced stochastic component in biological information 
pathways. Nevertheless, neural networks can store and retrieve information reliably. 
So noise is not a bug in biological systems, it is a feature [30, 31]. Nonetheless, 
although processes at the local neural level are highly stochastic, the entire nervous 
system exhibits rhythmic brain waves. In the human brain, they appear in the form 
of alpha, beta, and gamma waves [32]. Characteristic features, such as STDP [21, 
25, 33], stochastic firing and bursting of neurons in the hundred Hz range, recurrent 
network structures, and aspects of oscillatory synchrony in larger neuronal ensem- 
bles [34-43] are essential ingredients in biological-based information processing. 
Moreover, factors related to the close interaction of a nervous system with its envi- 
ronment, i.e., external stimuli, are of crucial importance [44]. Therefore, neuronal 
design principles provide a model for neuromorphic systems, which are diamet- 
ric to development strategies in present binary ICT, including precise GHz clock 
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frequencies, near-light-speed signal transmission, and clearly separated logic and 
memory [45, 46]. In contrast to clock-driven von Neumann machines [9, 47], infor- 
mation processing in biological nervous systems is characterized by highly parallel, 
energy efficient, and adaptive architectures [10, 11, 14, 48]. 

Now we turn back to the field of neuromorphic circuits [10—15]. Bio-inspired com- 
puting aims to realize biological plausible information pathways (a few are mentioned 
above) in engineered systems. However, this goal immediately leads to numerous 
questions and challenges: Which of the manifold and intriguing information path- 
ways observed in biology need to be mimicked by neuromorphic circuits to establish 
novel computing architecture with superior properties to conventional von Neumann 
systems? Could any biological process be simulated by a von Neumann computer to 
achieve similar performances as the biological model? Should novel neuromorphic 
circuits be made on today’s mature Si-CMOS platform or might it be wise to open 
the material “tools box” apart from Si technology in order to facilitate the integra- 
tion of novel devices and/or to incorporate self-assembly strategies similar to those 
observed in nature? 

Might it be beneficial to design circuits working at biologically plausible time 
scales, i.e., with pulse durations of about a few ms and signal conduction velocities 
of about m/s, which lead to strong signal retardations? Finally, yet importantly, the 
consideration of stochasticity could be an additional design goal in accordance with 
its fundamental role in biology. These questions display only a small selection of pos- 
sible approaches to mimic biological information pathways in engineered systems. 
This chapter focuses on neuromorphic circuits that take up those hallmarks of biologi- 
cal information processing that have been discussed above, i.e., STDP, stochasticity, 
oscillatory computing, and so on. The circuits compromise redox-based memris- 
tive devices as key components. Memristive devices consist of capacitor-like layer 
sequence, i.e., metal-memristive material-metal stacks. A universal property of the 
memristive devices is that the memristive state depends on previously induced charge 
flows, applied currents, or applied electric fields, thus storing any resulting resistance 
state. Memristive devices can be engineered to exhibit either binary switching or an 
analog resistance variation. Both device categories are subjects of this chapter and 
show beneficial features in pattern recognition and oscillatory computing. For details 
concerning resistive switching and the underlying physical-chemical mechanisms, 
we refer the reader to Chap. 3 and the overwhelming literature on the subject [49-52]. 


2 Requirements for Memristive Devices for Neuromorphic 
Computing 


Memristive devices are being explored for many different types of neuromorphic 
computation schemes, where their non-volatility allows computation to be per- 
formed in memory [53]. In this respect, memristive devices allow overcoming the von 
Neumann bottleneck, where memory and computation are separated from each other 
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Fig. 2 Classification of different biologically inspired computing architectures within time- 
dependent and time-independent computational schemes: the more biological the computational 
scheme, the higher the expected cognitive performance 


[9]. However, there are many different neuromorphic computing architectures, which 
place different requirements on memristive devices [54, 55]. Figure 2 shows a classi- 
fication that relates different network architectures to their biological inspiration and 
cognitive performance. While time-independent computing schemes are most widely 
used, time-dependent computing schemes are more biologically realistic. However, 
their technical implementation is much more challenging. While oscillatory comput- 
ing and spiking neural networks (SNNs) taking the temporal dimension of the signals 
into account [14, 56-58], mem-computing, and deep neural networks (DNNs) work- 
ing in a time-independent way using synchronized signals (i.e., they are based on 
clock signals) [57, 58]. This section will show the differences in the requirements for 
the memristive devices. For this purpose, a short overview of the architectures will be 
given, and the requirements they impose on memristive devices will be elaborated. 
Concrete examples of memristive networks that use time-independent computational 
schemes are discussed in Sect. 3, while examples of time-dependent neuromorphic 
architectures are presented in Sect. 4. 


Mem-Computing 


Mem-computing was originally invented for non-bio-inspired applications which 
aim to implement new computing architectures tasks like stateful [59, 60] and non- 
stateful logic [61, 62], in-memory arithmetic operations [63], solving linear and 
partial differential equations [64—66], optimization [67—69], and signal processing 
[70, 71] (for a review, the reader is referred to [3, 55, 58]). In all of those applications, 
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memristive devices are required which own a fast and low power switching, high 
cycling endurance (e.g., > 10!* cycles for stateful logic) as well as a low device- 
to-device and cycle-to-cycle variability [55, 58]. However, the inherent randomness 
of memristive switching mechanisms are challenging for high-precision computing 
tasks and different techniques are needed to increase the overall precision of the 
devices [64, 65, 69]. 


Deep Neural Networks (DNNs) 


DNNs [72] (including convolutional neural networks (CNN) [73]) are bio-inspired 
computing schemes that benefit from the in-memory computing architectures incor- 
porating memristive devices. In these networks, a large number of artificial neurons 
arranged in layers connected in a feed-forward structure by adjustable weights. These 
analog weights are trained with the backpropagation algorithm, which implements 
the delta rule between each neural layer [72]. These networks are specialized in 
pattern recognition tasks and build the backbone of today’s machine learning appli- 
cations [3, 58]. However, several drawbacks come along with DNNs. The networks 
are usually set up in software running on traditional von Neuman architecture, i.e., 
mostly on general-purpose graphics processing units (GPUs) [74] or tensor process- 
ing units (TPUs) [75]. Since a huge amount of data is needed for training and many 
learning cycles are required for real applications, these networks consume a large 
amount of energy and space. In addition, the training is very time-consuming [3, 57]. 
In this context, memristive devices can provide a solution as their in-memory comput- 
ing properties enable parallelization of processes, reducing power consumption and 
training time by orders of magnitude [76-79]. The significantly increased efficiency 
lies in implementing matrix-vector multiplications (MVMs) in hardware utilizing 
Ohm’s law and Kirchoff’s current law [54]. For this, however, strict requirements 
must be met by the memristive devices to be used as artificial synapses in DNNs, 
which are summarized in Fig. 3. A linear, gradual, and symmetric change in resistance 
is required for training [77, 80, 81]. For example, it has been shown that a 2% devia- 
tion from perfect symmetry increases the required number of analog states to train a 
DNN from 100 to 1000 [82]. In addition, endurance is essential since a lot of data is 
needed for training, combined with high energy efficiency and low latency to enable 
training directly on edge devices [83]. On the other hand, the training algorithm can 
compensate for device variability and yield to a certain degree [82]. Transferring 
pre-trained weights to memristive devices, furthermore, leads to less strict require- 
ments the devices have to fulfill. In this respect, it has been shown that a resolution 
of four to eight bits is sufficient to compete with floating-point precision weights 
in the inference process [75, 84]. For this purpose, multiple binary devices can be 
combined to mimic the weights of a synapse [85, 86]. However, the device variabil- 
ity and yield are more critical for pre-trained networks, and endurance becomes less 
crucial [82]. Retention is also of particular relevance. While for training, short-term 
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Fig. 3 Comparison of the requirements memristive devices have to fulfill to be suitable for different 
computational architectures 


retention and stability are sufficient [78, 82, 87], for inference, long-term retention 
and stability must be given [78, 82, 87]. 


Spiking Neural Networks (SNNs) 


In SNNs, the computation is based on synaptic connectivity and asynchronous, event- 
driven, and temporally precise signals [14, 15, 56-58, 88]. This enables such net- 
works to adapt to changing environmental conditions and react accordingly. How- 
ever, adequate local learning algorithms are required to exploit those advantages of 
SNNs over DNNs. These algorithms must satisfy the special needs of memristive 
devices and neuromorphic network structures in equal measure [89-97]. At the mem- 
ristive device level, several criteria, and in some cases different requirements from 
DNNs, must be met in order to satisfy the bio-inspired learning rules designed for 
SNNs. Figure 3 summarizes these and compares them to the device requirements 
of DNNs. Since nonlinear time-dependent conductivity modulations are the basis of 
learning in SNNs [57, 58], a gradual but nonlinear resistance change in memristive 
devices can be beneficial [57, 97]. Furthermore, both the endurance and energy- 
efficient switching and read-out are crucial properties for memristive devices in SNNs 
[57, 83]. However, device-to-device and cycle-to-cycle variability [57] and yield are 
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less critical since the effect of defective devices can be mitigated during learning 
[93]. Furthermore, the switching speed is expected to be less important [57, 88], 
especially if the networks run on a biologically relevant time scale with a spike dura- 
tion of milliseconds and frequencies of a few Hz [33]. The performance of SNNs is 
often investigated with pattern recognition tasks with frame-based datasets [3, 98]. 
In this respect, typical benchmark datasets containing static images are MNIST [99], 
CIFAR [100], and ImageNet [101]. These datasets do not contain temporal informa- 
tion. Thus, they do not allow to show the full potential of the time-dependent SNNs 
[102] and an outperformance in comparison to DNNs has not been reported so far 
[58]. More suitable benchmarks for SNNs should contain real-world spatio-temporal 
data, e.g., collected with event-based sensors [98, 103]. Thus, instead of executing 
pattern recognition on static data, SNNs are expected to be superior in interacting 
with the real world in a dynamically changing environment by processing continu- 
ous but sparse input streams on an energy-efficient way [57, 98, 102]. More suitable 
benchmarks for those tasks are dealing, e.g., with hand and arm gesture detection 
(DvsGesture) [104], automated driving [105, 106], or robotics [107]. Moreover, gen- 
erally applicable learning algorithms and network structures, which can cope with 
several different tasks, are within the focus of research [3, 15]. 


Oscillatory Computing 


Nature uses time-coherent dynamics for information processing based on the forma- 
tion of context-dependent, self-organized, and transient network structures. These 
enable us to react adequately to changing environmental conditions. Furthermore, 
these self-organized network structures are an important property for sensory integra- 
tion [36, 108]. Even if the underlying mechanisms are only partially understood, the 
interaction between dynamics and topology has been identified as one of the essen- 
tial building blocks of information processing in the brain in recent years [109]. In 
the current understanding, it is assumed that information is encoded into coherent 
states by temporally correlated neural activity patterns [110]. This concept offers, 
particularly, an elegant explanation for the binding problem - the question of the 
mechanism of sensory integration, which allows our brain to construct uniform per- 
ceptions from the multitude of sensory information. First evidence of these concepts 
could have been gathered from experiments with sensorimotor networks [111]. More 
recent studies have shown the universality of these concepts for the entire brain [110]. 
In this respect, memristive devices allow a new degree of freedom for the concept of 
neural synchrony: a local memory that supports a transient connectivity pattern [1 12— 
114]. The requirements for memristive devices needed for this have been less studied 
and are the content of Sect. 4.2. However, the inherent stochasticity of memristive 
devices has been shown to be helpful for this application [113]. 
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3 Time-Independent Neural Networks 


This section deals with two memristive networks that use time-independent compu- 
tational schemes, both relying on a fully CMOS-integrated 4 kbit resistive random- 
access memory (RRAM) array [115-117]. The first network discussed in Sect. 3.1 
is a mixed-signal-circuit implementation of a DNN for the detection of chronic 
obstructive pulmonary disease (COPD) [118]. Here, the devices are used to store the 
pre-trained weights of the DNN while the neurons are implemented in software. In 
that way, the possibility for on-chip recognition of saliva samples using in-memory 
techniques to detect COPD in a Point-of-Care application is shown. The second 
network introduced in Sect. 3.2 exploits the RRAM cells’ inherent stochasticity to 
solve a pattern recognition task [119, 120]. The stochastic artificial neural network 
(StochANN) is able to learn a subset of the MNIST benchmark through the adapta- 
tion of synaptic weights directly in hardware (in a mixed-signal realization) through a 
supervised local stochastic learning rule. Additional simulations of StochANNs with 
a larger number of devices show the performance limits of such a network for the 
whole MNIST benchmark. The results are compared to state-of-the-art approaches 
using time-independent and time-dependent networks. 


3.1 Deep Neural Network Implemented in CMOS-Integrated 
RRAM Arrays Used for Chronic Obstructive Pulmonary 
Disease Detection 


In this section, a time-independent DNN trained for disease detection is introduced 
[118]. The three-layer network is composed of a binary, fully CMOS-integrated 4 kbit 
RRAM array [115-117] emulating the synaptic weights, while neurons are imple- 
mented in software. The network was trained entirely in software, and the weights 
were subsequently transferred to the RRAM array. In that way, memristive devices 
can be used for inference, while they do not have to fulfill the same requirements as 
needed for training (see Sect. 2). In the following, first, the disease to be detected and 
the relevant input parameters for the machine learning (ML) method are introduced. 
Afterward, the network implementation and its performance are described. 

COPD, one of the most prevalent lung diseases worldwide, runs a perfidious course 
with an often long-lasting undiagnosed initial phase. Clinical treatment approaches 
for COPD result in repeated clinical visits and extended hospitalization for patients. 
This fact, apart from being an economic burden for healthcare infrastructures, drasti- 
cally impacts patients’ life quality. To address this issue, today’s healthcare systems 
have encouraged the development of personalized solutions through which patients 
can receive appropriate medical assistance in an outpatient clinic or a home-care 
environment [121]. Recent advances in point-of-care medical devices have facilitated 
the early detection, prevention, and treatment of various diseases [122]. However, 
without analytical insight, collected data from medical sensors are merely raw data 
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with low clinical value. For instance, in our previous work, a portable biosensor for 
managing COPD in home-care environments was presented [123]. The developed 
biosensor was capable of characterizing the viscosity of saliva samples for diagnostic 
purposes. However, saliva samples’ viscosity properties are one parameter of various 
parameters required for COPD detection. As a result, upon viscosity measurements 
by the developed biosensor, a sophisticated diagnostic algorithm is required to detect 
COPD by concurrent consideration of all essential parameters related to a patient’s 
personal and medical background. These demographic parameters include, but are 
not limited to, age, gender, weight, cytokine level, pathogen load, and the smok- 
ing background of subjects. Therefore, machine learning tools, or more specifically 
pattern recognition methods, could make the diagnostic procedure more efficient 
by converting collected data from medical sensors into meaningful clinical infor- 
mation. Moreover, machine learning can be used for identifying diagnostic links 
between symptoms and diseases that have been previously unknown and providing 
treatment plans and recommendations to healthcare specialists. 

As a result, implementing ML tools is crucial for converting collected raw 
data from subjects into meaningful clinical-diagnostic information. Furthermore, 
advanced ML analytics could make the management of COPD in Point-of-Care appli- 
cations more efficient. Nevertheless, drawbacks of cloud-based ML techniques for 
medical applications such as data safety, immerse energy consumption, and enormous 
computation requirements need to be addressed for this application. To address these 
challenges, CMOS-integrated RRAM arrays can be used for the hardware-based 
implementation of ML methods. Therefore, a memristive neuromorphic platform is 
presented in this work for on-chip recognition of saliva samples of COPD patients 
and healthy controls. Two groups of saliva samples, 160 for Healthy Controls (HC) 
and 79 for COPD patients, were collected in the frame of a joint research project at 
the Research Center Borstel, BioMaterialBank Nord (Borstel, Germany) [124, 125]. 
Patient materials were collected and anonymized prior to accessibility. The sampling 
procedure of the saliva samples was approved by the local ethics committee of the 
University of Luebeck under the approval number AZ- 16-167. Figure 4 demonstrates 
a hierarchy chart, categorizing the collected saliva samples into extended subgroups 
with respect to their diagnosis, gender, and smoking status [124]. As shown in Fig. 5, 
analog values of these four attributes were converted into 23 binary bits [gender (1), 
smoking status (3), age (9), dielectric permittivity (10)]. Dielectric sensors could be 
used to characterize sputum samples collected from patients for early diagnosis of 
COPD. The CMOS-based dielectric sensor system used for the real-time monitoring 
of sputum samples is described in [126]. 

The neuromorphic hardware implementation [118] of the developed ANN model 
was performed with a 4-kbit array of CMOS-integrated RRAM devices based on 
amorphous HfO, developed by IHP [115-117]. The array consists of 64 x 64 memris- 
tive cells in a 1-Transistor-1-Resistor (1T-1R) configuration. The two distinct states, 
low resistance state (LRS) and high resistance state (HRS), were used for the imple- 
mentation. The mean read-out currents are 30.8 mA and 3.2 mA at 0.2 V for LRS 
and HRS, respectively. For the deployment of the 10-level model, a mixed-signal 
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Fig.4 Hierarchical categorization of collected saliva samples into extended subgroups with respect 
to their diagnosis, gender, and smoking status. Reprinted from [124] (licensed under CC BY 4.0, 
https://creativecommons.org/licenses/by/4.0/) 
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Fig. 5 Conversion of analog attributes of the dataset (gender, smoking status, age, and dielec- 
tric properties) into 23 binary bits. Reprinted from [118] (licensed under CC BY 4.0, https:// 
creativecommons.org/licenses/by/4.0/) 
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neuromorphic circuit with software-based neurons and hardware synapses was used 
[119, 120]. Considering the topology of the ANN model (see Fig. 6) with one hidden 
layer and one read-out layer with four and two neurons per layer, respectively, 106 
parameters (i.e., synaptic weights and biases) were required for connecting the net- 
work layers. The resistance states of 1060 memristive devices on a single chip were 
set to the HRS or LRS, respective to the pre-trained weights. Every network param- 
eter is represented by the combination of ten devices where five devices represent 
positive values and five devices represent negative values, respectively. The sum of 
ten read-out currents at 0.2 V represents the total value of one synaptic weight. After 
successfully implementing pre-trained weights on the hardware, the test subset of 
data was used to evaluate the performance of the neuromorphic model for the recog- 
nition of COPD and HC samples. In order to recognize the COPD samples with the 
mixed-signal approach, the 23 input bits of the test-subset data were applied to the 
simulated neurons within the input layer. The output neurons of every subarray are 
perceptrons with a sigmoidal activation function, which receive the sum of current 
values passing through the connected devices together with a specific bias value. 
These current values are normalized to the maximum value of the pre-trained analog 
network to guarantee that the sigmoid function is activated with a reasonable range 
of values. The output values of the third layer (read-out layer) perceptrons denote 
whether a test sample belongs to COPD or HC categories. This hardware realization 
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agrees with the theory of neural networks that the weighted sum of inputs determines 
the value of a perceptron in the subsequent layer, as illustrated in Fig. 6. 

In summary, the concept of on-chip recognition of saliva samples of COPD 
patients using a memristive neuromorphic platform was studied. A hardware-friendly 
artificial neural network model was developed and trained for classifying COPD and 
HC samples using real clinical data. Subsequently, a 10-level conversion of the trained 
classification model was transferred onto a memristive neuromorphic platform for the 
on-chip recognition. The memristive chip provided a remarkable accuracy of 89%, 
offering an alternative approach to cloud-based methods required for diagnosing 
COPD in Point-of-Care applications. 


3.2 Stochastic Learning with Binary CMOS-Integrated 
RRAM Devices 


The inherent stochastic nature of the filament formation and dissolution in RRAM 
devices is challenging for many applications, especially if a high numerical precision 
is needed (see Sect. 2). On the other hand, different approaches benefit from the 
randomness of resistive switching and exploit it explicitly for the technical emulation 
of biological information processing. Such networks include noise tolerant stochastic 
computing technologies [127], synchronization of oscillatory neurons to emulate 
neuronal coherence [113, 128] as described in Sect. 4.2, stochastic switching neurons 
[129, 130] and stochastic learning rules realized with single binary synapses [119, 
120, 131-133], as well as compounds of several binary devices as one synapse [85, 
129, 131, 133]: 

The stochastic learning algorithm [119, 120] described in this section utilizes the 
stochastic nature of binary fully CMOS-integrated 4 kbit RRAM arrays [115-117] 
in a 1|-transistor-1-resistor (1T-1R) configuration, the same technology as used in 
Sect. 3.1, in a stochastic artificial neural network (StochANN) to learn the MNIST 
benchmark [99]. In that way, it is shown that the proposed StochANN is able to pro- 
cess analog information with binary memory cells. The devices are composed of a 
HfO,_,/TiOz_, bi-layer sandwiched between TiN electrodes. They can be switched 
between two distinct resistance levels, i.e., HRS and LRS. As an initial step, an 
electro-forming process is required. This is reliably done by the incremental step 
pulse with verify algorithm (ISPVA) [134]. The electrical device properties depend 
on the crystalline phase of the HfO2_, [120, 135]. The switching probability for 
polycrystalline and amorphous HfO2_, films is shown in Fig. 7a, b and c, d, respec- 
tively. The device-to-device (D2D) variability of 128 1T-1R devices is therefore 
determined by applying single voltage pulses in the set [Fig. 7a, c] and reset [Fig. 
7b, d] regime, i.e., for the transition from HRS to LRS and vice versa. To obtain 
the measured data shown as dots in Fig. 7, resistance states were measured with a 
read-out voltage of 0.2 V after applying a positive or negative voltage pulse for set 
and reset transition, respectively. A current of 20 mA has to be exceeded to count 
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Fig.7 Switching probability of the used RRAM devices dependent on the applied voltage amplitude 
of 10 ms pulses. Dots represent measured data, while the solid lines are fits with Eq. 1. The fit 
parameters d and Vo, as well as the switching window AV, are also depicted. In a, b, the set and 
reset behavior of the polycrystalline devices are shown, respectively, while in c, d, the set and reset 
behavior of the amorphous devices are depicted, respectively. For each technology, 128 devices 
were measured. Reprinted from [120] (licensed under CC BY 4.0, https://creativecommons.org/ 
licenses/by/4.0/) 


as an effective set operation, while the current has to be lower than 5 mA to count 
as a successful reset process. All pulses had a length of 10 ms. The cycle-to-cycle 
(C2C) variability shows no significant deviation from the D2D variability in similar 
devices [136]. Furthermore, the switching voltages determined here do not depend 
on the devices’ position within the 4 kbit array. Thus, taking the D2D variability 
into account for designing the learning rule is reasonable. The switching probability 
dependence on an applied voltage pulse can be described by a Poisson distribution 
taking voltage amplitude and pulse width into account [113, 119]. The distribution 
function for N voltage pulses (neural activity level) with a voltage amplitude V can 
be expressed as [113, 120] 
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Here, Vo denotes the voltage at which the probability fy is equal to 0.5, and d is a 
measure of the distribution functions slope and, therefore, of the switching variability. 
The larger the absolute value of d, the smaller the switching window AV,,, in which 
a stochastic encoding of analog data is possible. The switching window is defined 
as the voltage interval in which the switching probability fy is between 2 and 98 %. 
Fitting the measured D2D variability with Eq. 1 leads to the solid line in Fig. 7 as well 
as to the depicted d and AV,,, values. In summary, AV;,, is smaller for amorphous 
HfO_, than for polycrystalline HfO2_, devices due to the grain boundaries’ impact 
on the D2D variability and a more homogeneous defect distribution in amorphous 
hafnia films [135, 137]. Furthermore, AV,,, is smaller in the set transition compared 
to the reset transition for both technologies. 

To emulate synaptic plasticity, the activity A of a neuron is encoded in voltage 
pulses with amplitudes V within the switching windows: 


V=V,+A.-AYV, (2) 

with a 
=, 3 
K (3) 


Here, N is the number of action potentials arriving at a neuron in the time inter- 
val At, and V; is the lower bound of the switching window. Exploiting the whole 
switching window to map analog activity, i.e., analog data, to the stochastic nature 
of the switching event is possible by a proper choice of AV. In the following, the 
StochANN utilizing this local learning rule is described, and the influence of the 
switching window size on the learning performance is shown. 

The MNIST benchmark [99] of static visual patterns is used within this work. The 
learning data set contains 60,000 images of handwritten digits from 250 different 
writers. Each image consists of 28 - 28 greyscale pixels with 256-levels. Some rep- 
resentations of the ten included patterns (i.e., digits from zero to nine) are shown in 
Fig. 8. A test data set contains additional 10,000 images, which can be used to deter- 
mine classification accuracy. For the StochANN [119, 120] described in this section, 
averaged images are used, as shown in Fig. 9. These are obtained by calculating the 
average greyscale values of 100 randomly chosen representations of each pattern. For 
learning, the pixel intensities of every image are mapped to the interval [0,1] by divid- 
ing the values of every pixel by the maximum pixel value of the respective image. 
Learning takes place in a supervised manner in a time-independent StochANN. The 
network topology is illustrated in Fig. 9. Each pixel has one corresponding input neu- 
ron connected to every of the output neurons (one for each pattern sketched here) in 
a two-layer feed-forward configuration similar to earlier approaches [89-93]. Each 
connection is made by a binary stochastic synapse. The input neurons map the pixel 
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Fig. 8 Examples of MNIST images. Each row shows different representations of one pattern 


intensities into voltage pulses, inducing a switching event with respective probability 
for the connections to the dedicated output neuron. In that way, the trained synaptic 
connections, i.e., memristive devices, form receptive fields of the output neurons. 
Either set or reset transitions of both technologies are used for learning. If the set 
transition is used for learning, reset pulses are applied to each synaptic device prior to 
execute the learning rule corresponding to a low probability Psat. Accordingly, a low 
probability set pulse is used if the reset transition is exploited for learning. Thus, sat- 
uration effects are avoided. In that way, all training images can be used several times 
to train the network. It should be noted that the number of output neurons can vary 
for two reasons. First, if only a subset of the patterns is learned, fewer output neurons 
are necessary (one for each pattern). Second, each pattern can be learned by several 
output neurons to increase the network performance. The StochANN performance 
is evaluated experimentally with a mixed-signal circuit emulating the synapses in 
hardware using the fully CMOS-integrated RRAM arrays and neurons in software. 
Details about the circuit design can be found in [120]. Moreover, the network is sim- 
ulated without taking device variabilities and imperfections into account. Here, the 
stochastic learning rule is simulated by generating a random number r;,; uniformly 
distributed over the interval (0, 1) for every pixel j of every learning image i and 
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Fig. 9 Schematic visualization of the network structure (center) with 784 input neurons and 10 
output neurons connected with memristive devices as binary synaptic weights. The learning data 
(left top) and the test data (left bottom) are sketched as well as the output neurons activation function 
(right). Reprinted from [120] (licensed under CC BY 4.0, https://creativecommons.org/licenses/ 
by/4.0/) 
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the respective synaptic weight w; j is set to | if the pixel intensity is larger than r;, j. 
The simulations serve to determine the maximum possible network performance by 
omitting device imperfections. Furthermore, the number of available devices limits 
the experimentally realized network size. Thus, larger networks can be simulated to 
compare the stochastic learning algorithm to state-of-art networks. After learning, 
the classification accuracy can be evaluated by applying test images to the network. 
In experiments, 50 randomly chosen images of each pattern are used, while all 10,000 
test images are used in simulations. Therefore, the pixel intensities are binarized. For 
each test image, a threshold value proportional to a global constant c and the mean 
pixel intensity of that image is determined. Pixel intensities, which do not exceed 
the threshold, are set to 0, and the others are set to 1. The larger c, the fewer pixels 
are active, as shown in Fig. 9. Every test image is shown once to the network by the 
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input neurons, which induce a 0.2 V read-out pulse for pixel values of 1. The output 
neurons are modeled as perceptrons with an activation function 


I= ek Aout.i.m 


Fout = (4) 


1 + eK Aout, im j 


Here, k is a positive constant that defines the slope, as shown in Fig. 9 and Aout,i,m 
is the normalized activity of the input neurons for the test image i weighted by the 
synaptic connections Wj m of input neurons j to the output neuron m according to 


784 


1 
Aout,im = 784 : 2, Pi,j,bin © Wj,m, (5) 


where p;i j,bin is the binarized intensity value of pixel j being part of image i. The 
weights Wj m are determined as logical 1 if the read current exceeds 10 mA and the 
set transition is used for learning. If the reset transition is used, wj, m is assigned a 
logical 0 if the current is larger than 10 mA. Thus, Eq. 5 is valid for both cases. The 
output neuron, which receptive field re-samples the test image best, has the highest 
activation Aout, im, and associates the test image to the pattern it learned. If several 
output neurons are used to learn the same pattern, the sums of all activation functions 
belonging together are evaluated. A classification accuracy, named recognition rate 
in the following, is determined by calculating the percentage of correctly assigned 
test images. 

Two MNIST subsets were used first to compare experimental results with sim- 
ulations. One subset contains the digits “0”, “1” and “9”, while the second subset 
consists of “0”, “3” and “8”. Thus, the patterns differ more from one another in the 
first set compared to the second set, where the patterns have more pixels in common. 
In total, 3 - 784 = 2, 352 individual synaptic connections are needed. This num- 
ber of functional devices is selected from each type (polycrystalline or amorphous 
devices) of the 4 kbit chips, and they are randomly assigned to the output neurons. 
The parameters c = 4 (binarization of test images), k = 5 (slope of output neurons 
activation function in Eq. 4) and Psat = 35% (for to avoid saturation of weights) were 
optimized in simulations. Here, recognition rates (mean value and standard devia- 
tion) of 84.5% (44.6%) for the subset {0, 3, 8}, and 87.0% (44.8%) for the subset 
{0, 1, 9} were determined with 100 simulation runs for each subset with five learn- 
ing epochs each. It should be emphasized here that the algorithm converges within 
these five training epochs. Combining the results of both subsets, a recognition rate 
of 85.7% (+4.9%) is obtained. This is plotted in Fig. 10a as dashed line (mean 
value) and grey area (standard deviation). Receptive fields trained experimentally 
are shown in Fig. 11 for both device technologies, both subsets, and both state tran- 
sitions used for learning. It is obvious that both types of devices and both transitions 
can be used within the network to learn the respective patterns in hardware. Further- 
more, the switching windows size AV,,, affects the learned patterns. In particular, 
the smallest switching window corresponding to the set transition of the amorphous 
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Fig. 10 Recognition rates of the StochANN. a Combined experimental results for MNIST subsets 
{0, 1, 9} and {0, 3, 8}. Mean values and standard deviations of five experimental runs for each pattern 
(i.e., ten runs in total for each data point) are shown as black dots while standard deviations are given 
as error bars. Simulation results (100 runs for each subset) are given as dashed line (mean value) and 
gray area (standard deviation). The abbreviations “poly” and “am” denote the polycrystalline and 
the amorphous HfO2_-based devices, and “set” and “reset” denote the transition used to emulate 
stochastic plasticity. b simulation results for the whole MNIST dataset are shown (mean values and 
standard deviation of five runs with five learning epochs each). These results are achieved with a fixed 
activation function (black squares) and an adaptive activation function (green circles) of the output 
neurons. The recognition rates with directly written receptive fields are also shown for comparison 


(blue triangles). Reprinted from [120] (licensed under CC BY 4.0, https://creativecommons.org/ 
licenses/by/4.0/) 
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Fig. 11 Receptive fields of learned patterns in hardware. The read-out currents of the RRAM 
devices measured with 0.2 V are shown. In a-d, the used patterns are {0, 3, 8} learned with the set 
transition of polycrystalline devices a, the set transition of amorphous devices b, the reset transition 
of polycrystalline devices c, and the reset transition of amorphous devices d. In e-h, the used 
patterns are {0, 1, 9} learned with the set transition of polycrystalline devices e, the set transition 
of amorphous devices f, the reset transition of polycrystalline devices g, and the reset transition of 
amorphous devices h. Read-out currents are encoded in the pixel color as specified in i. Reprinted 
from [120] (licensed under CC BY 4.0, https://creativecommons.org/licenses/by/4.0/) 


devices leads to receptive fields, which are more challenging to differentiate visually 
[Fig. 1 1b, f] compared to the largest switching window belonging to the reset transi- 
tion of the polycrystalline devices [Fig. 1 1c, g]. However, the determined recognition 
rates show deviations from the simplified assumption that an increased switching 
window leads to better classification accuracy. In Fig. 10a, the recognition rates for 
both technologies and both state transitions are show. Each data point denotes com- 
bined mean values and standard deviations for five experimental runs of each subset 
(i.e., ten experimental runs in total for each data point). On the x-axis, “am” denotes 
amorphous and “poly” denotes polycrystalline HfO2_, devices. Furthermore, AV; 
is given in Fig. 10a. In summary, the medium-sized switching windows correspond- 
ing to the reset transition of amorphous HfO,_, devices and the set transition of 
polycrystalline HfO,_, devices show performances within the error margin of the 
simulations. Thus, the simulations accurately reproduce the experimental results. A 
larger switching window for the reset transition of the polycrystalline HfO2_, devices 
and a narrower switching window for the set transition of the amorphous HfO2_, 
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devices, however, result in worse recognition performance. Thus, evidence is shown 
that device variability has an impact on network performance. In particular, the size 
of the switching window must not be too small to optimize the stochastic synapses 
in the proposed StochANN. The reason for the worse accuracy obtained with the 
largest AV,,, has to be evaluated in the future. 

The network performance is tested with all ten patterns in simulations as well. All 
results were obtained with five learning epochs in five simulation runs. The recogni- 
tion rates are summarized in Fig. 10b. Black dots show the StochANN performance 
for a fixed slope k = 5 of the output neurons’ activation function. Recognition rates 
were determined as 53.3% (+3.0%) for ten output neurons (i.e., one for each pattern) 
61.6% (+1.9%) for 100 output neurons and 62.9% (+0.7%) for 300 output neurons. 
An increase in learning epochs or the number of output neurons does not lead to any 
improvement. Furthermore, these results are compared to patterns written directly 
into the synaptic states without any learning algorithm involved. For this purpose, the 
input patterns were binarized using a fixed threshold ©,;,. The synaptic weights were 
set to | if the corresponding pixel intensity of the input images were larger than the 
threshold. As shown by the blue triangles in Fig. 10b, a 75.7% recognition rate was 
obtained for a fixed Opin of 0.26, which was optimized in simulations. No standard 
deviation can be denoted since no stochasticity is involved but only one deterministic 
prototype of each pattern is stored. Thus, binarizing the input images and writing 
them directly into the receptive fields improves the performance compared to the 
stochastic learning rule. However, a thorough optimization of Opin for the specific 
dataset has to be performed, which becomes more tedious as the number of different 
input patterns increases. Moreover, an adaptive slope of the output neurons’ activa- 
tion function in combination with the stochastic learning rule leads to even higher 
recognition rates [green squares in Fig. 10b]. Here, the slope k is adapted with 


784 
5 Wim 
j= 
km = ko — Ak 784 (6) 
where ko is the base value, and Ak is a positive constant weighted by the total strength 
of the synaptic connections. Thus, the slope is steeper for neurons that have learned 
patterns with less active pixels, leading to a stronger activation of those neurons for 
less input strength, as can be seen in Fig. 9. The slope adaptation only depends on 
the final weights stored in the synaptic connection. No adaptation during learning is 
necessary. The adaptive slope has similar functionality to variable threshold values 
for the output neurons reported for other pattern recognition networks [89, 90, 92, 
93, 138, 139]. Here, the adaptive thresholds are essential to obtain a high recognition 
performance by emulating homeostasis. With kg = 5 and Ak = 6.8, a classification 
accuracy of 68.8% (+1.2%), 78.3% (41.2%) and 78.5% (40.2%) were achieved 
for 10, 100, and 300 output neurons, respectively. This shows that the stochastic 
learning rule slightly outperforms the directly written receptive fields for more than 
100 neurons in the output layer. This can be explained by the fact that only one 
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prototype of each pattern exists for directly written patterns, while variations of the 
prototypes exist in the learned receptive fields. 

Using more complex time-independent networks performing supervised learning 
leads to recognition rates > 98.5% for SNNs [140, 141] and 99.87% for a thor- 
oughly optimized CNN [142] and thus comparable to human performance estimated 
to be approx. 99.8% [143]. Using unsupervised learning in time-dependent neural 
networks is reported to achieve 93.5% with 300 output neurons [90] as well as 91.9% 
and 95.0% with 1600 and 6400 output neurons and the same amount of inhibitory 
neurons [138]. The latter approach was extended in Ref. [95] to a so-called lattice 
map (LM)-SNN leading to an accuracy of 94.07% for 1600 excitatory and inhibitory 
neurons, respectively. For a broad overview of different time-dependent and time- 
independent networks using supervised or unsupervised learning methods, the reader 
is referred to the overwhelming literature [98, 138, 142, 144-146]. All results named 
so far were obtained in simulations where only in [90] memristive devices were mod- 
eled to be used as synaptic connections. A fully hardware-implemented CNN based 
on multilevel RRAM devices can achieve recognition rates of 96.2% [147]. Here, 
a five-layer network is trained in software in a supervised manner, and the weights 
are subsequently transferred to eight 128 x 16 1T-1R arrays using two devices as 
one synapse to obtain positive and negative weights. Moreover, re-training of the 
last feature extraction layer was done in hardware. Another approach, in which two 
analog RRAM devices are used as one hardware synapse together with software 
neurons, reaches an accuracy of 91.7% for a re-scaled MNIST dataset of 8 - 8 pixel 
size. Here, a three-layer network using one array of 128 x 64 1T-IR devices can 
be utilized for learning directly in hardware using a supervised learning scheme 
[148]. Simulations of an extended network show a recognition rate of 97.3% taking 
device variability into account. A neuromorphic processor implementing a multi- 
layer SNN with static random access memory (SRAM) allowing on-line supervised 
learning reaches a recognition accuracy of 97.83% [149]. Furthermore, hardware 
acceleration of DNN inference with pre-trained weights transferred to PCM devices 
is reported to lead to a recognition rate of 98.3% [150]. Another integrated circuit 
utilizes memristive devices as synaptic connections with the possibility of on-line 
learning has also been published [151]. 

The StochANN performance shown here is promising for such a simple network 
structure but has to be improved to compete with other reported networks. One big 
drawback of the proposed concept is that only the averaged pattern can be learned. 
Transfer the supervised learning rule into an unsupervised learning approach can 
potentially help to extract more prototypes of each pattern [89, 90, 93] without the 
need for supervised learning by computing and gradually improving an error function, 
as done in classical backpropagation algorithms [77, 80, 81]. Furthermore, using 
several binary devices as one synapse can help to improve the network performance 
[85, 129, 131, 133]. The advantage of the proposed network is that learning can be 
done directly in hardware with a mature technology using fully CMOS-integrated 
RRAM devices as synapses. 
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4 Time-Dependent Neural Networks 


In this section, examples of time-dependent memristive networks are covered. In 
Sect. 4.1, an SNN based on analog memristive devices performing bio-inspired learn- 
ing and pattern recognition is presented [93]. Simulations reproducing real device 
behavior on learning the MNIST benchmark are provided while the impact of device 
variability and yield is investigated. A mixed-signal circuit implementation using 
real crossbar-integrated double barrier memristive devices (DBMDs) [152] to learn 
basal patterns experimentally is furthermore shown [94]. In Sect. 4.2, examples for 
oscillator computing with memristive devices are provided. According to Fig. 2, 
these networks show the highest degree of biological inspiration and, therefore, the 
highest amount of cognitive performance is expected. The oscillator-based com- 
puting scheme shown below emulates perception by transient synchronization of 
memristively coupled oscillators. Thereby, it establishes a certain analogy to biology 
to solve the binding problem [113]. The influence of the switching dynamics of two 
types of memristive devices on the synchronization of oscillators is, furthermore, 
investigated, and device requirements for oscillatory computing are deduced. 


4.1 Bio-Inspired Learning with Analog Memristive Devices 


In this section, a time-dependent neural network utilizing analog memristive devices 
to emulate bio-inspired learning for a pattern recognition task is presented. The net- 
work performance is investigated by simulations incorporating real device behavior 
[93] and by the realization of a mixed-signal circuit using real crossbar-integrated 
devices [94]. Here, LTP and LTD are induced by replicating the Hebbian learning rule 
described in Sect. 1 for unsupervised bio-inspired learning. Hebbian learning was 
already realized a decade ago with single memristive devices by emulating STDP 
[153, 154] as well as LTP and LTD [155]. 

The used devices are so-called double barrier memristive devices (DBMDs) with 
the layer sequence Au/Nb, Oy, (2.5nm)/Al)03(1.3 nm)/Nb [152] which are explained 
in detail in Chap.3. Here, memristive switching is reported to take place by field- 
driven oxygen ion movement within Nb,O,, modulating the effective Schottky bar- 
rier height and the effective tunneling width of the Al,O3 [152, 156]. Thus, ahomoge- 
neous interface-based switching leading to a gradual resistance change is performed. 
The amount of resistance change depends on the applied voltage amplitude and time. 
A mathematical description of experimentally determined switching data is given by 
the memristive plasticity model of Ziegler et al. [24]. This model is compatible with 
advanced biophysical plasticity models that can fit experimental data on STDP, while 
it is also suitable to describe plasticity emulation with memristive devices. In that 
way, a behavioral model is obtained, which can be used for network-level simulations 
to explore how the modeled devices can be utilized to emulate Hebbian plasticity in 
trainable neuromorphic networks. Therefore, the degree of conductance change, i.e., 
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the change of synaptic weight œw, is linked to the applied voltage pulses. The weight 
change is given by [24] 


dw 1 
— =B@)o(t) (1-- 0). (7) 


dt max 


Here, £ is the weight-dependent learning rate, and @max is the maximum achievable 
weight. The switching dynamics of memristive devices are expressed in £, which 
depends not only on the electrical stimuli but also on the present conductance state. 
Thus 6 depends on the switching mechanism of the memristive device and can 
lead to various learning behaviors [24, 157]. The learning rate can be different for 
potentiation 6, and depression fa. Furthermore, the synaptic weight œw represents 
the conductance G of a memristive device. Since the conductance change usually 
depends on the voltage pulse amplitude AV as well as the width Ar and the number 
of pulses n, the learning rates 8, and By are also modeled to be dependent on these 
parameters [24]: 


By (G,n, At, AV) = kpa (A VAAD (L — yG@ — 1)) (8) 
Ba (G,n, At, AV) = —kga(AV)A(At)yG(n — 1), (9) 


where kp, ka, and y are positive constants, while œ and A account for the non-linearity 
of the memristive devices’ switching process. 

Figure 12 shows the plasticity measurements (conductance vs. pulse number) 
of DBMDs investigated with voltage pulses of different amplitude and widths on 
86 single devices. Black dots denote the average data from 86 individual devices, 
while error bars denote the standard deviation. Red solid lines show a replication 
with the introduced plasticity model. Model parameters are given in the respective 
original paper [93]. The gradual conductance modulation is clearly visible in Fig. 12a. 
Here, 1000 equivalent positive voltage pulses inducing potentiation and subsequent 
1000 equivalent negative voltage pulses inducing depression were applied to the 
devices, as illustrated in the inset. A pulse duration of At = | ms was chosen together 
with AV = 3.9 V and AV = —2.5 V for potentiation and depression, respectively. 
Device conductance was read out by applying a voltage of 0.48 V, i.e., well below the 
threshold to change the device state [152], after every 100 potentiation or depression 
pulses. The data are depicted relative to the average maximum conductance G max = 
100 nS of all devices after 1000 potentiation pulses. To determine the variations of 
the device conductance in dependency on the pulse amplitude and width, potentiation 
pulses with AV between 2.4 V and 3.7 V and a fixed width of 1 ms [Fig. 12b] and At 
ranging from 1 ms to 30 ms and a fixed amplitude of 3.9 V [Fig. 12c] were used. The 
data points in both figures show device conductance after 1000 pulses. The impact 
of depression pulses is shown in Fig. 12d for 1000 voltage pulses of 30 ms length, 
and AV between -1.4 V and -2.6 V applied to previously fully potentiated devices. 
An asymmetry between positive and negative voltages is obvious. Moreover, the 
device conductance is nearly unaffected for positive voltages of 2.4 V and below 


68 F. Zahari et al. 


e Measurement = Plasticity Model 


l (>) 
s . 0.1Ẹ 
E g F 
F o F 
F Goor, 
£ FY At=ims N=1000 4 
oof, ...,° i i ee | 
0 1000 2000 2.5 3 3.5 4 
Pulse number AV (V) 
bi omer Ty 
1Ẹ E Ẹ 
50.1 9 01Ẹ 
me : O f 
H AV =3.9V N=1000 j i At = 30 ms N=1000 
eee sees eeeeeee ‘eeere veewe ee! 0.01 A eee wae es weet 
O 5 10 15 20 25 30 -3 -2.5 -2 -1.5 -1 
At (ms) AV (V) 


Fig. 12 Plasticity measurements of DBMDs: Mean values from 86 individual devices are given 
as black dots, while error bars denote the standard deviation. Conductances are normalized with 
the average maximum conductance Gmax achieved with 1000 potentiation pulses of AV = 3.9 V 
amplitude and At = 1 ms width. Fits with the plasticity model are given as red solid lines. a 
Typical conductance modulation with 1000 potentiation pulses (AV = 3.9 V, At = 1 ms) and 1000 
depression pulses (AV = -2.5 V, At = | ms) is shown. Each data point depicts the normalized 
conductance measured after 100 pulses. The conductance change after 1000 potentiation pulses 
with varying pulse amplitudes and widths is shown in b and c, respectively. The conductance 
change with 1000 depression pulses of different amplitude is shown in d. Reprinted from [93] 
(licensed under CC BY 4.0, https://creativecommons.org/licenses/by/4.0/) 


as well as for negative voltages with an absolute value of 1.4 V or below. By using 
potentiation pulses with AV = 3.9 V, however, the conductance can be increased 
by two orders of magnitude, which can be fully turned back with AV = —2.6 V.A 
thorough analysis of the data is given in [93]. 

The measured data incorporated in the plasticity model can now be used to sim- 
ulate a neuromorphic network capable of learning visual patterns by adjusting the 
synaptic weights emulated by DBMDs. The MNIST dataset [99] of handwritten dig- 
its, which is introduced in Sect. 3.2, shall be learned. The network operates similarly 
as networks reported in other works [89-92]. The two-layer feedforward network 
is schematically shown in Fig. 13 [93]. Here, each input neuron (blue circles) is 
connected to every output neuron (red circles) by DBMDs (symbols of memristive 
devices) arranged in a crossbar array. Every input neuron stochastically encodes the 
intensity of one pixel into voltages pulses [91—93]. Therefore, the pixel intensities are 
normalized to the interval (—1, 1). The absolute values of the normalized intensities 
denote the probability of an input voltage pulse generation, while the sign stands 
for the voltage polarity. In that way, every input neuron either generates no spike or 
voltages pulses of +0.6 V or -0.6 V amplitude, both not affecting the device conduc- 
tance by themselves. The currents flowing to the output neurons depend not only on 
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Fig. 13 Schematic of the simulated neural network. Reprinted from [93] (licensed under CC BY 
4.0, https://creativecommons.org/licenses/by/4.0/) 


the voltage pulse amplitudes but also on the conductance of the devices. The leaky 
integrate-and-fire (LIF) output neurons [33, 158] integrate the incoming stimuli until 
a certain threshold is reached [89]. When an output neuron reaches its threshold, a 
voltage pulse consisting of a positive part with AV = 2.9 V and a negative part with 
AV = —2.3 V. This post-synaptic pulse overlaps with the pre-synaptic pulses and 
changes the conductance of memristive devices connected to the respective output 
neuron. If a positive pre-synaptic pulse superimposes with the post-synaptic pulse, 
a potentiation takes place for the positive part (Vsum = 3.5 V) while the negative 
part does not affect the memristive state significantly (Vsum = —1.7 V). Vice versa, 
a net depression takes place when a negative input pulse overlaps with an output 
pulse (Vsum = 2.3 V and V;um = —2.9 V, respectively). If no input pulse occurs, the 
post-synaptic pulse’s voltage amplitudes alone do not significantly impact device 
conductance. Thus, input pixels with a strong intensity lead to positive input pulses, 
which, superimposed with induced output pulses, increase the device conductance. 
Pixels with low intensity induce negative input pulses, which lead to decreased con- 
ductance if an output spike simultaneously occurs. Thus, prototypes of the patterns 
are stored in the resistance states of the memristive devices. All devices connected 
to an output neuron are building the receptive field of this specific neuron. In that 
way, unsupervised associative learning based on local Hebbian plasticity is realized. 
Essential for network performance is, furthermore, an inhibitory coupling network 
for the output neurons implementing a winner-takes-it-all (WTA) mechanism, in 
which the first spiking neuron resets the integration of all other neurons [89]. More- 
over, an adapting individual threshold is implemented for the output neurons to allow 
that all output neurons participate equivalently in learning. This mimics homeostasis 
in biological systems [89]. 
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Fig. 14 Obtained receptive fields after unsupervised learning with 50 output neurons. In the gray- 
scale used to visualize device conductance white indicates maximum device conductance (strong 
synaptic weight), while black represents minimum conductance values (weak synaptic weight) of 
the memristive devices. Reprinted from [93] (licensed under CC BY 4.0, https://creativecommons. 
org/licenses/by/4.0/) 


After learning, the network can be used to classify unknown images. Therefore, 
the output neuron, which receptive field matches the input image best, creates an 
output pulse. To assign every output neuron to its learned pattern, a small amount of 
pre-classified images is applied to the network and it is evaluated for which pattern the 
output neurons get activated. Afterward, the network performance can be evaluated 
by applying all 10,000 images from the MNIST test dataset to the network and 
calculating recognition accuracy by determining the percentage of correctly assigned 
patterns. Therefore, only the pre-synaptic pulses to encode the images are used, 
and the spiking events of the output neurons are tracked while post-synaptic pulse 
generation is suppressed to stop changing the device states. For 10, 20, 50, and 
100 output neurons, recognition rates of 65%, 70%, 77%, and 82%, respectively, 
were determined. These rates are in good agreement with similar networks [89-92]. 
A typical set of learned receptive fields obtained in a simulation with 50 output 
neurons is shown in Fig. 14. It can be seen that the implemented network using the 
Hebbian learning scheme is able to learn different prototypes of all ten patterns (digits 
from zero to nine). The obtained performance is significantly lower than those from 
other spiking networks, as described in Sect. 3.2. However, network requirements 
for using memristive devices in SNNs are examined in this work, while improving 
pattern recognition computing schemes was not intended. 

The results presented so far were generated without taking device variability 
into account. Now, the impact of different degrees of D2D and C2C variability is 
investigated [93]. Therefore, networks with ten output neurons were trained with 
three iterations of the whole MNIST learning dataset. Results are shown in Fig. 
15(a,b), in which every data point was determined as the average of three simulation 
runs. The findings are then compared in Fig. 15c to the measured variability of real 
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Fig. 15 Impact of device variability, i.e., variability of local learning rate of individual devices, on 
network performance. Device-to-device (D2D) and cycle-to-cycle (C2C) variability are depicted 
as standard deviation of Gaussian distributions. Each data point represents the mean value of three 
simulation runs. a D2D variability b C2C variability for a D2D variability of 0 % (black dots), 40 % 
(red squares) and 80 % (blue diamonds) ¢ Experimentally determined D2D variation for potentiation 
with AV = 3.9 V and At = 1 ms. Each data point shows the normalized mean conductance of 86 
devices after 100 potentiation pulses. The red, blue, and gray lines indicate the range of learning 
rates with, 40%, 80%, and 100% D2D variability, respectively. Reprinted from [93] (licensed under 
CC BY 4.0, https://creativecommons.org/licenses/by/4.0/) 
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DBMDs. To model D2D variability, the learning rates of all devices were varied with 
Gaussian distributions [see inset of Fig. 15a] initially in every simulation run. As 
depicted in Fig. 15a, a standard deviation of up to 50% does not affect the recognition 
rate, while a further increase in variability slightly influences the performance. The 
C2C variability was modeled by varying the individual devices’ learning rate for 
every applied image with a Gaussian distribution. Figure 15b (black dots) shows 
that a C2C variability of up to 200% does not affect the network performance. A 
combination of D2D and C2C variability, the most realistic scenario, is given in 
Fig. 15b with red squares and blue diamonds. Robust performance is achieved for 
a C2C variability of 100% combined with a D2D variability of 80%. Furthermore, 
the numerical investigation provides evidence that the most crucial performance 
losses result from a constant D2D variability since this effect does not average out 
in many learning iterations like it is the case for C2C variability. To estimate if 
the D2D variability of DBMDs does allow to use them as artificial synapses in the 
investigated network, the measured D2D variability is compared to the theoretically 
obtained boundaries. Figure 15c shows experimentally recorded device conductance 
of 86 individual devices after every 100th potentiation pulse with amplitudes of 3.9 V 
and 1 ms normalized by the highest recorded conductance G max,totai. Furthermore, 
the solid lines in Fig. 15c indicate the variation range of learning rates with 40%, 
80%, and 100% D2D variability. Thus, the experimentally obtained D2D variability 
lies within the required variation interval. Moreover, in the original paper [93], the 
experimentally obtained yield, i.e., the percentage of functional devices, is shown to 
be approx. 98%, which does not influence the network performance significantly. In 
conclusion, evidence is provided that DBMDs are attractive candidates to be used as 
artificial synapses in neuromorphic circuits. In particular, the gradual conductance 
change under voltage pulsing, as well as the variability and yield of such real devices 
are believed to be suitable for the investigated network and learning rule. 

The possibility of using DBMDs as artificial synapses has also been shown exper- 
imentally [94]. Here, a two-layer network, like described above, has been imple- 
mented in a mixed-signal-circuit. The synaptic connections were emulated with a real 
crossbar array containing 16 - 16 = 256 devices. Due to the high Z-V non-linearity 
and the diode-like character of the DBMDs, no additional selector devices are needed 
to avoid the sneak path problem (see Sect. 2 and Chap. 3). The Z-V characteristics 
of crossbar integrated devices were proven to be similar to single devices [94], as it 
is also shown in Chap. 3. The neurons were emulated in software. During learning, 
the conductances measured with a pre-synaptic pulse of 0.9 V amplitude were used 
to compute the current flowing to the output neurons. If a post-synaptic spike is 
triggered, potentiation and depression pulses with AV = 3.6 V and At = 100 ms as 
well as AV = —1.1 V and At = 300 ms, respectively, were used. In that way, the 
overlapping of pre- and post-synaptic pulses was not realized, but the potentiation and 
depression took place according to the Hebbian learning scheme explained above. 
This deviation from simulations was needed because the simple circuit design did 
not allow for real parallel data processing [94]. However, the work aimed to provide 
a proof of principle that DBMDs can be used in a crossbar configuration without the 
need for additional selector devices as artificial synapses in neuromorphic networks. 
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Figure 16a shows the simple 6 - 6 pixel pattern that the network shall learn. Figure 
16b shows the developing receptive fields for five output neurons. Every pixel rep- 
resents the resistance of one memristive device emulating the synaptic connection 
between one input and one output neuron. Initially, the resistances encoded in the 
pixel color are randomly distributed. After using 22,000 learning images in total, all 
three patterns are learned. Thus, the realization of unsupervised bio-inspired learn- 
ing was possible with real DBMDs arranged in a crossbar structure thanks to the 
high Z-V non-linearity and the diode-like character, as well as due to no required 
initial electro-forming step. As shown in the simulations above, the presented system 
can, in principle, cope with more complex tasks. However, a much larger amount of 
memristive cells is necessary for that. Due to the high resistances of DBMDs (even 
in LRS) and the several orders of magnitude lower wiring resistance (~ 100 Q for 
an individual wire with the size 1100 - 40 - 0.5 mm?) in the present crossbar array, 
larger arrays are believed to work as well. 


4.2 Oscillatory Computing 


Biological information processing relies heavily on nonlinear dynamics [36, 109]. 
This enables the integration of the multitude of information in an enormous and 
massively parallel network of neurons divided into functionally specialized regions 
such as the visual cortex, auditory cortex, or dorsolateral prefrontal cortex. Each 
of these regions participates as a context-dependent, self-organized, and transient 
subnetwork [36, 108]. Even if the underlying mechanisms are only partially under- 
stood, the interaction between dynamics and topology has been identified as one of 
the essential building blocks of information processing in the brain in recent years 
[109]. In the current understanding, it is assumed that information is encoded into 
coherent states by temporally correlated neural activity patterns [110]. This concept 
offers, particularly, an elegant explanation for the binding problem - the question 
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of the mechanism of sensory integration, which allows our brain to construct uni- 
form perceptions from the multitude of sensory information. First, evidence of these 
concepts could have been gathered from experiments with sensorimotor networks 
[111]. More recent studies have shown the universality of these concepts for the 
entire brain [110]. Recently, oscillators coupled by memristive devices have been 
shown to emulate this kind of information processing to some extent [112, 113]. 

In this section, we report on oscillator computing with memristive devices. We 
show how the dynamics of oscillator networks, coupled by memristive devices, is 
affected by the resistance of these devices. Therefore, important requirements for 
memristive devices are discussed as well as applications with the possibility to open 
up new pathways towards the construction of cognitive electronics. 


4.2.1 Oscillator Computing with Memristive Connectivity 


From studies of the thalamocortical system, Hoppensteadt and Izhikevich proposed a 
computational scheme based on oscillators with different frequencies that are weakly 
coupled to an externally changed medium, causing dynamic connectivity [159]. In 
their model, information is encoded in the oscillators’ phase and/or frequency syn- 
chrony. The weak coupling, thereby, allows a dynamic change of their connectivity 
patterns depending on an external signal. A similar approach follows the idea of 
memristive coupled oscillator structures [112, 113], which will be explained in the 
following. 

Figure 17 shows the model of two memristively coupled oscillators. Both oscil- 
lators are initially oscillating in their own frequencies f; and f; [Fig. 17a]. As long 
as their coupling is weak, they are not affecting each other. However, if the coupling 
strength between the oscillators increases, they start to interfere and synchronize 
in frequency and phase for sufficient high coupling strengths [Fig. 17b]. However, 
if the coupling strength is decreased thereafter, the oscillators will desynchronize 
again due to their different frequencies. This model can be realized with two self- 
sustained van der Pol oscillators with resistive coupling, as shown in Fig. 17c [112], 
i.e., via the conductance g,, of memristive devices. While in the following the model 
of memristive coupled oscillators will be discussed in the framework of van der 
Pol oscillators, any other type of oscillator may also be suitable [160]. The oscilla- 
tor system shown in Fig. 17 can be described by the following set of second-order 
dimensionless nonlinear equations: 
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Fig. 17 a, b Concept of memristively coupled oscillators. c Depending on the coupling strength 
8m (conductance of the memristive device) a frequency and phase synchronization of the oscillators 
occurs, as seen from the time course of the oscillator voltage yı (y2) in the lower graph 


Here, $, y , and « are positive constants that define the uncoupled oscillators’ damp- 
ing, non-linearity, and frequency behaviors, respectively. Furthermore, gm is the 
mutual coupling, representing the conductance of the memristive device. The mem- 
ristive device can be modeled via 


I = gm(x,t)-V_ with a = f(x, V,t). (12) 
Here, x is the memristive state variable, V the voltage across the memristive device 
(for example, V = yı — y2), and f a dynamic function describing the voltage-driven 
atomic reconfiguration in the particular material system [160]. The device con- 
ductance influences the coupling strength via g,,(x,t) = x - Go, + (1 — x) - Gore, 
where Gon and Gopr are the maximum and minimum conductance of the device, 
respectively. The obtained result is shown in Fig. 17c. Here, the change of the cou- 
pling strength [upper graph in Fig. 17c] synchronizes the initially asynchronous oscil- 
lators. Thus, memristively coupled oscillators allow emulating the initially described 
essential principle of biological information processing: synchronization (informa- 
tion encoding into coherent states by correlated neural activity) and memory (change 
of connectivity). 
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4.2.2 Memristive Devices for Oscillator Computing 


Using suitable memristive devices is an important point for realizing the previously 
presented model in hardware. The typical characteristics of memristive devices can be 
roughly divided into two classes. The first class of devices shows an abrupt resistance 
jump at a specific voltage (type 1), while the second class exhibits a gradual resistance 
change under an applied voltage (type 2). In Fig. 18, representatives of the two 
classes of memristive devices are compared. The first device (type 1) has the layer 
sequence Al/TiO,/Ag and is an electro-chemical metallization cell (ECM), while the 
second device (type 2) is composed of the layer sequence TiN/TiO,/HfO,/Au, and 
its resistance change is functionally based on a valence-change mechanism (VCM). 
For the latter, a bi-layer oxide structure has been used, which is known to stabilize 
the resistance switching mechanism [161, 162]. Figure 18b shows measurements for 
the two types of devices. While for type 1 cell an abrupt jump in the resistance is 
observed, and the investigated type 2 device shows a more gradual transition of the 
conductance under voltage cycling. To analyze the consequences of the device type on 
the dynamics of coupled oscillators, two scenarios were simulated: (i) an abrupt and 
fast change of the coupling strength, when a threshold voltage is reached (behavior 
type 1) and (ii) a gradual change of the coupling strength, when the threshold value 
Vin,p is exceeded (grey line in Fig. 18). Furthermore, the bi-directional switching 
behavior was taken into account such that gm is reduced again when the voltage falls 
below a negative threshold value V;;,,,. The obtained results are shown in Fig. 18c. 
It can be seen that the binary switching devices show a much faster synchronization. 
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Fig. 18 a Typical memristive devices with archetypal switching characteristics. b /-V character- 
istics of an Al/TiOx/Ag cell (type1) and TiN/TiOx/HfO,/Au (type2). c Simulation of the dynamics 
of the two coupled oscillators (yı and y2) via a memristive device (conductance gm) with an abrupt 
change in resistance (red) and a gradual change (blue). Here yı — y2 represents the voltage across 
the memristive device. The threshold values for set and reset of the devices are shown in gray 
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4.2.3 Experimental Realization and Networks 


A hardware implementation of memristively coupled oscillators is shown in Fig. 19. 
Here, van der Pol oscillators are realized via a programmable unijunction transistor 
(PUT) based circuit, as described in detail in Ref. [112]. In this circuit, the two 
oscillators are coupled via a memristive device so that their voltage oscillations (at 
points u; and u;) are smaller than the thresholds of the devices. Thus, the voltage 
oscillations cannot change the resistance value of the memristive device [see Fig. 
19a]. The ECM cell shown in Fig. 18a was used as the memristive device, which 
changes its conductance gm depending on an external voltage signal ujearn. AS shown 
in Fig. 19b, the oscillators can be synchronized with this circuit depending on gm. 
For the example shown in Fig. 19b, a voltage sequence of seven voltage pulses of 
2.2 V amplitude was applied to the memristive device. This yields a conductance 
change of the device. 

The transition to a simple artificial neural network is shown in Fig. 19c and d. 
The four oscillators of the first layer are connected to the two oscillators a and 
b of the second layer. Initially, all six oscillators of the network oscillate in their 
own frequency, as indicated in the contour plot by the different colors [Fig. 19c]. By 
applying a voltage sequence ujearn, the connectivity matrix gn between the oscillators 
changes, which strengthens the coupling between oscillators 1, 2, and a as well as 
oscillators 3, 4, and b. As a result, the system oscillates with only two frequencies, 
as shown in Fig. 19d (different colors in contour plot). This computing scheme 
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Fig. 19 a Experimental realization of memristively coupled van der Pol oscillators. b Voltage 
courses uj and u; are labeled in the circuit diagram in a. The voltage train ujearn (starting at 
100 ms) decreases the resistance of the memristive device and the previously asynchronous oscil- 
lators synchronize. The ECM cell shown in Fig. 18a was used as memristive device. c Network of 
6 van der Pol oscillators. By changing the coupling matrix gj as a consequence of altered external 
conditions, the oscillators are synchronized in two frequencies (red and blue) d 
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was used in Ref. [113] to emulate perception. Thereby, Uiearn can be identified as a 
level of perception that strengthens or weakens different connections depending on a 
temporal change of perception. For that, the inherent stochasticity of the memristive 
devices was used. In this respect, the number of voltage pulses defines the probability 
that the memristive device changes its resistance state. Thus, the number of voltage 
pulses of ujearn simulates the level of perception. Therefore, a certain analogy to 
biology can be established in the solution of the binding problem [113]. 
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can be engineered, which enables tailoring of the memristive devices for particular 
applications. 
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1 Introduction 


The term “Memristor” is a portmanteau from the two words “Memory” and “Resis- 
tor’. A memristor (also called a memristive device) typically consists of capacitor- 
like layer sequences, i.e., a metal-insulator-metal structure. A universal property of 
the memristive device concept is that the memristive state depends on previously 
induced charge flows, applied currents, or applied electric fields, which result in dif- 
ferent stored resistance states. This simultaneously unique and simple device prin- 
ciple, together with the predicted potential for breakthrough technologies in areas 
such as universal memories and novel non-Boolean computing schemes for cog- 
nitive electronic systems, drives the research and development of memristors and 
memristor-based circuits. Memristive devices were intensively studied in the last 
century’s sixties and seventies [1-3]. The field was further driven forward by the 
establishment of the theoretical background of memristors by Leon Chua (1971), 
with the corresponding experimental realization and interpretation by the Hewlett- 
Packard (HP)-Labs (2008) [4, 5]. 

Currently, two main development avenues can be explored for memristive devices. 
The first focuses on resistive random access memories (RRAMs). It is believed that 
the zoo of today’s existing memory diversity can be replaced by a single (universal) 
memory concept. RRAMs are considered attractive candidates for universal memo- 
ries because they: show non-volatile data storage, can be densely integrated, are fast, 
and are cheap to produce. In particular, such a universal memory might attenuate the 
problem known as memory latency in modern digital computers [6, 7]. Besides the 
RRAM goal, which may be categorized under the label “More Than Moore”, novel 
and very appealing computer architectures have been proposed in which memris- 
tors might play a vital role. Another main spotlight of possible memristive device 
applications may be assigned to such catchphrases as non-Boolean computing, bio- 
inspired information processing, neuromorphic engineering, or cognitive electronics 
[8—16]. On the local synaptic level, learning in nervous systems is explained by the 
Hebbian learning rule [17] and, amongst others, spike-timing dependent plasticity 
(STDP) [18]. STDP and other memory-related mechanisms observed in nervous 
systems, such as long term potentiation (LTP) and long term depression (LTD) [19], 
were successfully mimicked by memristive devices [8, 20-22]. To what extent larger 
networks of memristive devices can mimic higher brain functions is still unknown. 

Until now, a huge number of experimental findings on memristor devices con- 
sisting of a wide variety of metal/insulator material combinations have been pub- 
lished, all of which show memristive Z-V curves [23]. For details concerning resistive 
switching and the underlying physical-chemical mechanisms, we refer the reader to 
the overwhelming literature on the subject [12, 14, 23-27]. These physical-chemical 
mechanisms include electronic effects (e.g., charging of electron traps) [25, 26], 
magnetic effects (e.g., spin-transfer torque - STT) [28] or ferroelectricity (e.g., ferro- 
electric tunnel junctions - FTJ) [29]. Moreover, nano-ionic mechanisms are widely 
utilized to build memristive devices. Included are phase change memory (PCM) 
devices [30] in which the resistance of the active layer is modulated by switching 


Redox-Based Bi-Layer Metal Oxide Memristive Devices 89 


Fig. 1 Schematic a 
illustrations of VCM devices ( ) (b) 
based on filament formation — metal— 


(a) and homogeneous ion 
movement (b) 


oxide 


=—— metal" 


oxygen oxygen ions or 
vacancies vacancies 


between crystalline and amorphous phase. Moreover, devices based on the forma- 
tion and dissolution of conductive filaments composed of metal ions, i.e., the elec- 
trochemical metalization effect (ECM) [31], or composed of oxygen vacancies, i.e., 
the valance change mechanism (VCM) [23, 32], belong to this class. Furthermore, 
VCM can also be realized without conductive filaments. In these interface-based 
memristive devices, the resistance is changed by modulating interfacial properties 
like the Schottky-barrier height [24, 32—34] or the tunneling probability [35-37] by 
the movement of oxygen vacancies or oxygen ions [23, 25]. Thus, the functionality 
of VCM devices is based on redox reactions. A sketch of a filamentary-based device 
and an interface-based device are shown in Fig. la and b, respectively. This chapter 
deals with VCM devices composed of bi-layer metal oxides. 

Since the requirements the devices have to fulfill depend on the application, both 
types of devices are investigated for different purposes, as described in more detail 
in chapter Neuromorphic Circuits with Redox-Based Memristive Devices and in fur- 
ther literature [23, 38—41]. In this respect, filamentary-based devices are available 
in a more mature technology compared to interface-based devices [41]. However, 
the need for an initial electro-forming step, a current compliance for switching, a 
pronounced intrinsic randomness of the switching process, and a limited amount of 
achievable resistance states are some drawbacks compared to interface-based devices 
[24, 38-42]. On the other hand, endurance, retention, switching speed, and scalabil- 
ity are some of the reported benefits of filamentary devices [38—42]. The properties 
of interface-based devices will be discussed in further detail in Sect. 2 while the 
properties of filamentary-based devices are described in Sect. 3. 

The quality of the solid-state electrolyte, in which ion movement occurs, deter- 
mines the switching properties. In this regard, it has been theoretically shown that 
an oxygen-deficient HfO, with x between 1.5 and 1.75 is required for the most effi- 
cient nucleation of vacancies as the starting point for filament growth [43]. Park et 
al. [44] experimentally verified this by showing that x = 1.8 leads to filamentary 
switching while x = 1.98 leads to interface-based switching. However, stoichiomet- 
ric HfO2 matrices are also reported to show filamentary switching enabled through 
local reduction to HfO2_, [45, 46]. Similar observations are reported for SrTiO. 
[47]. However, in these devices, only the filament-type switching is explained by the 
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movement of vacancies. In contrast, the interface-based switching is explained by 
the charging and discharging of electron traps. Moreover, it has been shown that the 
oxygen content in HfO, [44, 45] and TaO, [48] can be used to modify important 
device parameters in filamentary devices, such as forming and switching voltages, 
memory window, and high resistance state (HRS) and low resistance state (LRS) 
currents. Furthermore, adjusting the switching window by tailoring the defect den- 
sity has also been reported for interface-based devices in simulation studies [44, 49]. 
Engineering device properties will be discussed in more detail in Sect. 4. The second 
oxide in the bi-layer system can have different functions, like acting as an oxygen 
reservoir or a diffusion barrier. A reservoir like TiO, [50, 51], TaO, [37], or a TION 
formed at a TiN electrode interface [52, 53] allows to increase or decrease oxygen 
vacancy or ion concentration in the solid-state electrolyte while performing mem- 
ristive switching. On the other hand, a diffusion barrier like Al,O3 [54—57] restricts 
ion movement within the electrolyte and, thus, can improve retention times [55] or 
switching linearity [56], and can lead to multi-state operation [56, 57]. It should be 
noted here that bi-layer metal oxide memristive devices are not necessarily switching 
by ion migration. As a second switching mechanism, the charging and discharging 
of electron traps are reported to be responsible for memristive switching in metal 
oxide junctions [58-63]. 

Both interface-based and filamentary-based VCM devices consisting of bi-layer 
metal oxides are described in the following, and exemplary realizations are shown in 
Sects. 2 and 3, respectively. Strategies to engineer the electrical parameters of these 
memristive devices to meet the requirements for specific applications are provided 
in Sect. 4. 


2 Interface-Based Devices 


The current transport is homogeneously distributed across the electrode area in 
interface-based memristive devices. Here, the resistance is determined by the inter- 
facial properties of the incorporated materials. Thus, the resistance change is based 
on altering interfacial properties like Schottky-barrier height or width [24, 32-34] 
or tunneling probability [35-37]. This is achieved by a rearrangement of oxygen 
vacancies or oxygen ions [23, 25] or by charging and discharging electron traps [58- 
60, 62]. In that way, interface-based memristive devices show analog, i.e., gradual 
switching [24, 35, 41]. Moreover, no electro-forming is needed, and devices with a 
high Z-V non-linearity [33, 64] to avoid the need for additional selector elements for 
passive crossbar integration [65] are reported. Furthermore, the intrinsic randomness 
is less pronounced than in filamentary devices [38, 41]. Moreover, interface-based 
VCM devices have already been integrated into 3D stacks in complementary metal- 
oxide-semiconductor (CMOS) technology with a minimum feature size of 5 nm 
[64]. However, CMOS integration and scaling to the nanoscale is challenging for 
most reported devices [41]. CMOS integration is challenging if the incorporated 
materials or the needed deposition steps are not used in CMOS fabrication lines or if 
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the used materials do not withstand the CMOS fabrication steps. This is explained in 
detail in chapter Integration of Memristive Devices into a 130 nm CMOS Baseline 
Technology. Downscaling is challenging because the resistance of interface-based 
devices scales with the electrode area, and most reported devices have high resis- 
tances even on the micrometer scale. Drawbacks are, furthermore, a worse retention 
time and slower switching speed [41] compared to filamentary devices. Therefore, 
interface-based devices are good candidates to emulate synapses in bio-inspired neu- 
romorphic networks [11, 40], as described in detail in chapter Neuromorphic Circuits 
with Redox-Based Memnristive Devices. Two different devices are described in this 
section. The first device is based on a Nb, Oy/A1203 bi-layer in which Nb, Oy, acts as 
the solid-state electrolyte while Aly Oz is a diffusion barrier restricting ion movement 
in the Nb, Oy layer [55]. The second device is based on a HfO2/TiO, bi-layer. Here, 
TiO, acts as an oxygen reservoir, enabling an oxygen exchange with HfO, [44]. 


2.1 Devices Based on NbxOy/Al,03 


The first device is named double barrier memristive device (DBMD) [55]. The dou- 
ble oxide layer Nb, O,/Al203 is sandwiched between an Au top electrode and an Nb 
bottom electrode, as it is sketched in Fig. 2a. The Au leads to a Schottky barrier for- 
mation, while alumina acts as a tunnel barrier. The movement of oxygen ions within 
Nb, Oy under an applied electric field is reported to be responsible for memristive 
switching [55, 66], as modeled by a kinetic Monte Carlo simulation [67] which is 
explained in more detail in chapter Modeling and Simulation of Silver-Based Fila- 
mentary Memristive Devices. It should be noted that the charging and discharging 
of electron traps as the switching mechanism cannot be ruled out [55]. However, 
a variation in the charge distribution within Nb, Oy leads to both a variation of the 
Schottky-barrier height and effective tunneling width. By scaling the solid-state elec- 
trolyte down to 2.5 nm, i.e., to the length scale of a single electron wave, the impact 
of the bulk resistance is minimized, and a very strong mutual interdependency of 
both barriers is achieved. Thus, both interfaces are altered simultaneously when the 
charge distribution varies. 

Devices with active areas between 70 pm? and 2500 pm? are produced on 100 mm 
Si wafers, which are passivated by a thermal SiO, layer. All layers are deposited by 
DC magnetron sputtering in one and the same vacuum chamber without breaking 
the vacuum. The devices are structured using standard optical lithography in com- 
bination with lift-off and etching steps. Additional passivation with SiO, enables 
the deposition of Ti metal contact pads, which allow an automatic measuring of 
device properties across the whole wafer. This will be discussed in more detail in 
Sect. 4.1. The fabrication process is explained in detail in [55, 66]. First, Nb and 
Al are deposited in an Ar atmosphere. The subsequent deposition of Nb,Oy from a 
metallic target in an Ar/O2 atmosphere leads to the fully oxidation of Al to Al203, as 
observed in electron energy loss spectroscopy (EELS) and energy-dispersive X-ray 
spectroscopy (EDX) in a transmission electron microscope (TEM) [69]. The Nb, Oy 
stoichiometry was found to be between the oxidation states Nb.O5 and NbO3. All 
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Fig.2 Double barrier memristive devices (DBMDs). a Schematic material stack. b Typical |J | -V 
hysteresis curve. c Area-dependent current transport in HRS (red) and LRS (blue). d Optical micro- 
scope photography of a 16 - 16 crossbar array containing DBMDs. e Typical |/|-V curves of a 
crossbar-integrated device (red) and a single device (black). f Retention of a DBMD (black) and 
a device without an Al203 layer. b, f are reprinted from [55] (licensed under CC BY 4.0), ¢ is 
adapted from [55] (licensed under CC BY 4.0), and d, e are reprinted from [68] (licensed under CC 
BY 4.0, https://creativecommons.org/licenses/by/4.0/) 
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interfaces but the Nb/Al,O3 interface appeared to be smooth on the nanoscale. Due 
to the roughness at the lower interface, the thinnest parts of the alumina were deter- 
mined to be 1.3 nm, as sketched in Fig. 2a. Furthermore, the interface between the 
Nb bottom electrode and Al,O3 was also slightly oxidized. The TEM measurements, 
including EELS and EDX, are described in more detail in chapter Critical Discussion 
of Ex situ and In situ TEM Measurements on Memristive Devices. 

Figure 2b shows a typical current density versus voltage (J-V) hysteresis curve 
of a DBMD, which was measured with a grounded Nb electrode, while voltage was 
applied to the Au electrode. Switching from HRS to LRS is taking place gradually, 
i.e., in an analog way, by applying a positive voltage to the Au top electrode. Switching 
back to HRS takes place by applying a negative bias to the top electrode. No initial 
electro-forming is needed, and an intrinsic current compliance is present. Both HRS 
and LRS show an area-dependent current transport, as shown in Fig. 2c, since the 
product of device area A and resistance R results in a constant value for both states. The 
absolute value of the current density is depicted on a log scale in Fig. 2b, showing the 
high J-V non-linearity and asymmetry. For dense device integration in a crossbar 
array, a selector device in series to every memristive element or a strong built-in 
I-V non-linearity is needed to avoid parasitic currents, i.e., the sneak path problem, 
when addressing a specific device [65, 70]. The built-in diode-like characteristics 
of DBMDs allows crossbar integration without needing additional selector devices 
[68]. A photograph of a crossbar array consisting of 256 DBMDs is shown in Fig. 2d. 
Here, the devices are located at the cross points of the metallization lines. In that 
way, each top and bottom contact pad connects the top electrodes in one column 
of the array. Each left and right pad connects the bottom electrodes of one row. 
Thus, only 32 contact pads are needed to address all 256 memristive devices. The 
I-V characteristics of crossbar integrated devices are similar to those of reference 
devices, as shown in Fig. 2e, in which the black line belongs to a reference device 
and the red line to a crossbar-integrated device, respectively. Furthermore, Fig. 2f 
shows the retention time of a DBMD. After the device was set from Roff (HRS) 
to Ron (LRS), the resistance was recorded every 60 s with a voltage pulse of 0.5 V 
amplitude and 2 s duration. In that way, a non-destructive read-out of the resistance 
state was performed, and the relaxation of the state was measured. The resistance 
increased fast in the first 600 s (black data points), while afterward, the resistance 
change was significantly less pronounced. An Ron/ Rosp ratio of more than one order 
of magnitude was still present after one day. This behavior can be quantitatively 
described by fitting the experimental data to a ~ tf power law (red lines) [61]. In the 
first 600 s, the relaxation can be described by ~ t%65 while afterward, ~ 1°!8 holds. 
That two different power laws are dominant for different time intervals suggests 
that different physical processes may be responsible for memristive switching, e.g., 
charging of traps and movement of ions on a faster and slower timescale, respectively. 
It is also shown in Fig. 2f that the Al,O3 considerably affects the retention time. 
The grey squares show the same measurement on devices without alumina. The 
retention characteristics of those devices show no resistance increase for the first 700 
s but a steep increase afterward fitted by a ~ £7? power law. Thus, the Al2O3 does 
significantly increase the retention time. Furthermore, the fact that alumina acts as a 
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diffusion barrier for oxygen [54, 56, 57] is also believed to lead to the self-limited 
switching behavior due to the limited amount of oxygen ions within Nb,Oy and no 
oxygen exchange between both oxides. 

The retention time of DBMDs does not allow to use these devices for classi- 
cal memory applications but for neuromorphic circuits [11]. Bio-inspired learning 
of visual patterns utilizing crossbar integrated DBMDs as hardware synapses have 
already been demonstrated [68]. This is explained in chapter Neuromorphic Circuits 
with Redox-Based Memristive Devices in detail. However, the drawbacks of DBMDs 
are those named above for interface-based devices [41]. In this respect, the scalability 
is challenging due to the high resistance, even on the micrometer scale. For quadratic 
devices with a feed size of 10 um, the current in HRS for a typical read-out voltage 
of 0.5 V is in the pA range. Thus, for scaling the devices down to the nanoscale, 
the resistance has to be tailored by orders of magnitude. On the other hand, high 
ohmic devices are desired for low-power applications. Thus, the resistance has to be 
adjusted so that currents are high enough to be suitable for electronic circuits but low 
enough to account for energy efficiency. Strategies for engineering device perfor- 
mances are given in Sect. 4. Furthermore, compatibility with CMOS technology is 
crucial for developing integrated circuits. While Al203 is a well-known dielectric in 
CMOS technology, Nb, Oy in the needed stoichiometry might be more challenging 
to be integrated into state-of-the-art fabrication lines. Moreover, gold is not desired 
in CMOS fabrication. Also, the impact of other fabrication steps (e.g., tempering in 
back-end-of-line processes [71]) has to be considered. CMOS integration of memris- 
tive devices is explained in chapter Integration of Memristive Devices into a 130 nm 
CMOS Baseline Technology. However, if CMOS integration is not possible, devices 
can be fabricated directly onto CMOS processed substrates in other fabrication lines 
[72, 73]. 

After describing device performance, including benefits and drawbacks of the 
introduced DBMDs, the model for current transport and memristive switching is 
explained in more detail in the following. Figure 3a shows an equivalent circuit 
emulating the /-V characteristics [55, 66]. Here, the Au/Nb, Oy interface is modeled 
as a Schottky diode D,, the Nb,Oy is modeled as a parallel circuit of a resistor 
Rwoo and a capacitor Cybo, and the Al2O0; layer is modeled as a voltage-controlled 
current source 7, emulating elastic tunneling in parallel to a capacitor Cazo. The 
variable describing the memristive state x denotes the average ion position within 
the Nb, O,. More precisely, thermionic emission theory was employed to model the 
Schottky diode [74, 75]. According to this theory, charge carriers have enough energy 
due to an applied electric field to cross the potential barrier between a metal and a 
semiconductor. For an applied forward bias V, this current can be calculated by 


Is = In (e7 — 1), (1) 


where q is the elemental charge, kg is the Boltzmann constant, T is the temper- 
ature, and n is the ideality factor. If n > 1, deviation from ideal thermionic emis- 
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Fig. 3 Model of a DBMD [66] a Equivalent circuit b Schematic of the band diagram. ©2016 IEEE. 
Reprinted, with permission, from [M. Hansen et al., IEEE International Conference on Rebooting 
Computing (ICRC) (2016)] 


sion theory is indicated. This deviation can be caused by, e.g., additional tunneling 
through the barrier (thermionic field emission), additional interface states, or barrier 
in-homogeneity [74, 75]. The reverse current Ig is given by 


® 
Ip = AAT e 5". (2) 


Here, A* is the effective Richardson constant, A is the device area, and Ọpg is the 
Schottky barrier height. For reverse bias, however, and if the apparent barrier height 
@z is reasonably smaller than the band gap of the insulator, the reverse current 
decreases gradually with the applied negative bias. This can be described by 


Iry = —A"AT2¢~ are aT 3 
R,V <0 = e Te T, (3) 


The device-dependent parameter œ, is used to fit the experimentally observed reverse 
voltage dependency. As it is described in [55], the Schottky diode dominates the 
current transport for voltages below 0.5 V (LRS) and 1.0 V (HRS). For higher 
voltages in the forward regime, i.e., higher positive voltages on the Au electrode, the 
Al,O3 tunnel barrier gets relevant and acts as in intrinsic current compliance. The 
tunneling current can be modeled by the Simmons equation [76] 
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In Eq. 4 holds x1 = ./@ — eV; /2 and x2 = ./® + eV;/2. Here, ® is the tunnel 
barrier height, and Vz is the resulting voltage across the tunnel barrier. Furthermore, A 
is the device area, K is a constant value, dox is the barrier thickness, anda = 2./2m /ħ 
(m: free electron mass; A: Planck’s constant divided by 27). 

As it is indicated in Fig. 3a, the Schottky barrier height ®g, the effective tunnel 
barrier width deff (corresponds to d;,, in Eq. 4) as well as the resistance of Nb, Oy 
Rwoo are dependent on the state variable x, i.e., on the average ion position. Thus, 
redistribution of oxygen ions within the Nb,Oy leads to a change in the interfacial 
properties of the Schottky barrier and the tunnel barrier, while the resistance of Nb, Oy 
is only slightly affected [55]. Details about the evolution of x under applied bias and 
the impact on the interfacial properties can be found in [55, 66]. This model was 
adapted by Dirkmann et al. [67]. Here, a lumped element circuit model containing 
the Schottky diode and the tunnel barrier is consistently coupled with a 3D kinetic 
Monte Carlo model for the ion transport. The simulation results show that the drift of 
charged oxygen ions within the Nb,O, can account for resistive switching behavior. 
In that way, a Schottky barrier height lowering from 0.90 eV to 0.83 eV, a decrease 
of the ideality factor from 4.0 to 3.4, and a decrease of the effective tunnel barrier 
thickness from 1.3 nm to 1.2 nm were determined to explain the Ron/ Roff ratio of 
more than one order of magnitude. The model also covers the retention behavior. 
Furthermore, the simulation indicates that the whole applied voltage drops across 
the Schottky barrier for reverse bias and low forward bias, while the tunnel barrier is 
responsible for the built-in current compliance at higher forward biases. The model 
is also described in more detail in chapter Modeling and Simulation of Silver-Based 
Filamentary Memristive Devices. A sketch of the band diagram for HRS and LRS is 
given in Fig. 3b and shows the decrease of the Schottky barrier height as well as the 
tunnel barrier width for switching from HRS to LRS [66]. 


2.2 Devices Based on HfO2/TiOx 


Another device based on the oxide bi-layer HfO2/TiO, sandwiched between an Au 
top electrode and a TiN bottom electrode is discussed now [44]. All used materials 
but the Au are already integrated into CMOS fabrication lines [77]. A typical Z-V 
characteristic for a device with an area of 100 um? is shown in Fig. 4a with grey 
dots. The switching mechanism is reported to be similar to that of the DBMDs. 
A Schottky barrier is formed between Au and HfO2. Mobile oxygen ions within 
HfO, lead to resistive switching, as sketched in Fig. 4b, c. However, as for the 
DBMDs, charging and discharging of electron traps cannot be omitted as a reason 
for switching [44]. The Z-V characteristics are qualitatively similar to the DBMDs. 
Thus, gradual and electro-forming free switching for voltages exceeding a certain 
threshold and a high Z-V non-linearity leading to a diode-like behavior are present. 
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Fig. 4 Interface-based TiN/TiOx/HfO,/Au device. a Typical |/| -V hysteresis curve. b Schematic 
material stack with indicated switching model. Red and blue spheres indicate mobile oxygen ions 
and stationary oxygen vacancies, respectively. c Equivalent circuit d Retention of interface-based 
devices (red) and filamentary devices (blue). Reprinted from [44] (licensed under CC BY 4.0, 
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However, one difference is that a current compliance of 10 pA is used for switching to 
prevent dielectric breakdown. Furthermore, the switching window, i.e., the difference 
between HRS and LRS, decreases with increasing device area. This leads to the fact 
that the HRS shows an area-dependent current transport while the LRS seems to be 
not area dependent. However, as it is described in detail in [44], the drift velocity 
of oxygen ions is modeled to be area dependent, which leads to less switching for 
increasing device areas and thus to a decreased R - A. The retention time can be fitted 
with a power law function of ~ t°, depicted in Fig. 4d by the red dots. 

The devices are produced with DC magnetron sputtering of all materials without 
breaking the vacuum and subsequent standard optical lithography and edging steps 
on 100 mm Si wafers passivated with thermal SiO2. Device areas between 100 and 
2500 um? are fabricated on one and the same wafer. One clear advantage of this 
material system is that the stoichiometry of HfO, can easily be adjusted during 
sputtering. A HfO1.98, which is named HfO> in this chapter for simplicity reasons, 
leads to the described device performance, while HfO;.s leads to filamentary-based 
switching, which is covered by Sects. 3 and 4.2. 

The device model shown in Fig. 4c contains a Schottky diode formed between 
Au and HfO2, which is described by thermionic emission theory (see Eqs. 1-3). 
The oxides are both modeled by parallel RC circuits. The model described in detail 
in [44] reproduces the Z-V characteristics very well, as depicted in Fig. 4a by the 
red solid line. Here, the resistive switching is dominated by Schottky-barrier height 
decreasing from 0.71 eV to 0.61 eV and the ideality factor increasing from 3.9 to 
4.45 by considering oxygen ion movement as switching mechanism. The model also 
covers the decreasing R - A with increasing A. 


3 Filamentary-Based Devices 


Many transition metal oxide-based memristive devices show resistive switching asso- 
ciated with filamentary-type VCM [23, 32, 78]. VCM is triggered by the migration 
of field-assisted oxygen ion-related defects in transition metal oxides. The oxygen 
ion-related defects are typically oxygen vacancies that are much more mobile than 
the transition metal cations [79]. The oxygen vacancy migration and valence change 
of the cation sub-lattice often appear in the form of the formation and dissolution 
of a conductive filament in a localized area in the oxide leading to filamentary-type 
switching. 

The resistive switching is typically observed after an initial electro-forming step in 
filamentary-type devices [79]. Double-positively charged oxygen vacancies (V;' *) 
are generated by electro-forming, and the Vj's migrate under the applied electric 
field. If Vj‘ *s are close to each other, a single charged state (V,") becomes more 
favorable. The cluster of Vis are arranged in a filament structure, where the VA 
has a significantly higher energy barrier for diffusion than V)'* [80]. The local- 
ized conductive filament leads to reversible changes of local resistivity and overall 
device resistance [81]. The vacancies located in the filament have a high diffusion 


Redox-Based Bi-Layer Metal Oxide Memristive Devices 99 


energy barrier. Therefore, the filamentary-type devices typically show long retention 
properties. 

The bi-layer metal oxide devices typically consist of an oxygen vacancy reser- 
voir layer and a solid-state electrolyte layer for filamentary-type memristive devices 
[82]. Under an applied electric field, oxygen vacancies are injected into the solid- 
state electrolyte from the reservoir layer. The oxygen vacancies form a localized 
conductive filament in the electrolyte layer, which reduces the overall resistance. If 
the electrolyte layer builds a Schottky-like barrier at the interface to the contacting 
metallic electrode, the conductive filament leads to a lowering of the barrier height 
and, thus, the overall device resistance [50]. The filamentary devices can be recog- 
nized by the resistance-area product [83]. The resistance in the ON state (LRS) is 
independent of the electrode area size because the conductive filament is created in 
a localized form. 

The benefits of filamentary RRAM devices are manifold. These devices can show 
a high endurance, high retention time, a fast switching speed (up to ns regime), and 
great scalability in the nm regime [38—42]. Moreover, they have already been inte- 
grated into CMOS technology, allowing to combine traditional electronic circuits 
with RRAM devices on the very same chip [77, 84-87], and even 3D integration has 
been shown [84, 88, 89]. Drawbacks are the need for an initial electro-forming step 
and a current compliance for switching, the need for selector devices to integrate 
them in memory arrays, pronounced intrinsic randomness of the switching process, 
a limited amount of achievable resistance states, and a rather low resistance [38—42, 
65, 70]. However, a bi-layer metal oxide concept has recently attracted attention 
to tackle the intrinsic randomness issue in filamentary devices [57]. Besides, ran- 
domness can be exploited for certain tasks [90, 91]. Moreover, even though most 
RRAM devices can only be switched between two resistance states (binary devices), 
multilevel devices have already been reported. For this multi-state operation, a vari- 
able current compliance for switching to LRS and a variable switching voltage for 
switching to HRS are usually needed [45, 52, 92—94], or variable pulse widths can be 
used [45]. However, devices showing a gradual switching with identical pulses have 
also been reported [56, 95]. Moreover, more sophisticated programming algorithms 
are reported to reach specific resistive states[57, 85, 87, 88, 94, 96, 97]. 

In this section, a TiO,-HfO, bi-layer oxide device is mainly presented as an 
example of a filamentary-based VCM memristive device [44]. Using the TiO,-HfO, 
bi-layer oxide device, typical electrical characteristics of filamentary-based devices 
are shown. 

The TiO,-HfO, memristive devices consist of TiN(50 nm) as the bottom elec- 
trode, Ti0,(30 nm)/HfO,(2-8 nm) metal oxide bi-layers, and Au(50 nm) as the top 
electrode. The devices were fabricated on a 4-inch wafer, where the devices have 
6 different active area sizes with a thickness gradient of the HfO, layer from 2 nm 
to 8 nm. This 4-inch wafer fabrication process enables to exclude wafer-to-wafer 
variability, and study the effect of the area size and the thickness of HfO, on the 
electrical properties of the memristive devices. 

In Fig. 5, the TiO0,/HfO, bi-layer memristive devices show a typical J-V curve as 
a filamentary-type device. At first, an electro-forming step was required to initiate the 
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resistive switching in the TiO,/HfO, memristive device. The electro-forming process 
was characterized by an abrupt jump in the current at a higher voltage (2.3 V) than the 
SET/RESET voltages. However, a high voltage through electro-forming could make 
it difficult for memristive devices to be integrated into CMOS circuits. Strategies to 
overcome this issue are already reported [45, 98, 99] and are addressed in Sect. 4.2. 

After the electro-forming cycle, the resistive switching is observed in the 
TiO,/HfO, memristive device. The switching required external current compliance 
of 5 mA for the SET process, while the RESET exhibited a compliance-free behav- 
ior. Bipolar switching properties were observed, and the amplitudes of switching 
voltages were symmetric, with —0.7 V and 0.7 V for SET and RESET, respectively. 
The TiO,/HfO, device featured gradual switching characteristics, while typical fil- 
amentary type devices show an abrupt SET process. The origin of the abrupt tran- 
sition during SET is related to positive feedback between the current increase and 
Joule heating [100]. In order to achieve a gradual switching behavior, a layer stack 
modification [100] and a multi-layer stack approach [51] have been proposed. In 
the multi-layer stack concept, for example, the TiO,/HfO, bi-layer oxide can be 
exploited, where TiO, serves as a reservoir for oxygen vacancies and stabilizes the 
switching process [57]. A gradual switching can promote uniformity during device 
operation. Furthermore, a gradual resistance transition during both SET and RESET 
is favorable for analog computing applications [101]. 

Gradual switching is often related to analog behavior. The analog behavior can be 
investigated using pulse measurements, applying a train of voltage pulses to mem- 
ristive devices. Figure 6a shows the switching behaviors of the TiO,/HfO, device 
in a pulse measurement. For the measurement, 20 SET voltage pulses followed by 
20 RESET voltage pulses are applied, while a reading pulse is applied after each 
switching pulse to read the resistance of the device. The TiO,/HfO, device shows 
a non-linear transition from the SET to the RESET, as shown in Fig. 6a. However, 
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Fig. 6 a A train of 20 set voltage pulses and the subsequent 20 reset voltage pulses. A non-linear 
transition from the SET to the RESET is observed. b Linear trend in the resistance change in 
dependency on the amplitude of the switching pulse voltage. c Multi-state in the SET and d in the 
RESET resulting from the adjusted amplitude of switching voltages. b is reproduced from [44] 
(licensed under CC BY 4.0, https://creativecommons.org/licenses/by/4.0/) 


the magnitude of resistance change is influenced by the amplitude of the applied 
switching voltage pulses in Fig. 6c, d. The impact of pulse amplitude is observed 
by applying switching voltage pulses with varied amplitude, each followed by 200 
reading pulses. Furthermore, the median values of resistance were extracted from 
Fig. 6c, d, and the resistance change from the initial state was calculated at each 
switching voltage amplitude. The resistance change showed symmetry in Fig. 6b 
between the SET and the RESET operation. Symmetric and analog resistive switch- 
ing is desirable for neuromorphic computing since this makes the programming of 
the memristive device more efficient [102]. 

Another indication of the filamentary-type device can be the retention prop- 
erty. In Fig. 4d, the retention property of the TiO,/HfO, device is compared to the 
interface-type device described in Sect. 2. The interface-type device has the same 
material configuration but fewer oxygen vacancies in the HfO, layer (x = 1.98). The 
filamentary-type device shows a much longer retention time than the interface-type 
device due to the high diffusion barrier of the vacancies within the filament. This has 
been quantitatively shown by fitting the retention measurement to the ~ tê power 
law [61] introduced in Sect. 2, leading to 6 = 0.02 compared to 6 = 0.3 for the 
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interface-based devices. Thus, the relaxation of the resistance is proceeding orders 
of magnitude slower in the filamentary devices. 


4 Engineering Z-V Characteristics of Memristive Devices 


The possibility of adapting electrical parameters of memristive devices allows engi- 
neering the device performance to meet the requirements for specific applications 
[45, 49]. Parameters such as switching voltage, forming voltage, HRS, LRS, the 
switching dynamics, or the number of resistance states can be engineered. Typical 
strategies are tailoring the switching oxide itself by adjusting the stoichiometry or the 
density of oxygen vacancies or ions [44, 45, 47-49], doping with additional cations 
[94], or using different oxide phases [91, 94]. Furthermore, adding additional oxide 
layers to affect the oxygen exchange [55-57] or the local temperature distribution 
[51] can impact the memristive behavior. Geometrical parameters like the switching 
oxide thickness [44, 71] or the device area [44, 55] can also influence the Z-V charac- 
teristics. In Sect. 4.1, strategies for tailoring the interface-based devices introduced 
in Sect. 2 are described. This includes a thorough analysis of the process plasma 
during sputter deposition of DBMDs. Section 4.2 deals with engineering approaches 
to tailor the filament-based devices shown in Sect. 3. 


4.1 Engineering of Interface-Based Devices 


The devices described in Sect. 2 are fabricated by magnetron sputtering [103, 104]. 
The thin film deposition conditions like pressure, gas composition, structure and 
strength of the magnetic field, amplitude and form of discharge, and substrate position 
and orientation strongly influence electrical and optical material properties [105-— 
109]. In this respect, gas flow rates and the resulting pressure significantly impact 
particle energies and, thereby, the structure and density of deposited films [106, 
107]. Furthermore, the gas composition affects chemical reactions in the gas phase 
and at surfaces (target, substrate), influencing, e.g., the stoichiometry. Moreover, the 
substrate position and orientation relative to the sputter target affect film properties 
since particle fluxes and electric potentials in the plasma environment can show 
strong spatial in-homogeneity due to, e.g., structure and strength of the involved 
magnetic field [105, 107—109]. Another crucial parameter during reactive sputter 
deposition in an electronegative gas like O? is the density of negatively charged 
ions. If the target surface gets completely poisoned, negative oxygen ions are formed 
and accelerated by the electric field towards the substrate. This can cause defects or 
stress in deposited materials due to the high kinetic energy [105, 110]. The radial 
distribution, the energy, and the impinging angle of these ions depend on the erosion 
of the target (i.e., the target geometry) and the discharge voltage. It has been shown 
that the erosion state of a target can be correlated with the resistivity of oxides, 
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Fig.7 Electrical characterization of DBMDs at different positions on a 100 mm wafer: a Resistance 
map measured with 1.6 V. b Device resistance dependent on radial position for positive bias (black 
dots) and negative bias (red squares). c-f) Typical |J|-V curves at different radial positions 1 to 
4 (—25 mm, +5 mm, +30 mm, and +35 mm). Reprinted from [49] [F. Zahari et al., Journal of 
Vacuum Science & Technology B 37, 061203 (2019)], with the permission of AIP Publishing 


attributed to the impact of negatively charged oxygen ions [105, 111]. The erosion 
is almost only taking place in a ring-shaped area (called race track) determined by 
the magnetic field. 

To correlate the plasma properties to the electrical characteristics of DBMDs, 
several plasma parameters of the Nb,Oy deposition process were measured at 
different positions underneath the Nb target since the J-V characteristics vary at 
different positions on the 100 mm wafer [49]. Figure 7 shows the device-to-device 
variation across the whole wafer. Figure 7a contains a resistance map measured 
with read-out voltages of 1.6 V on a spatial resolution of 1 mm, showing circular 
symmetry. Here, 5336 devices with an area of 400 um? were measured. Figure 7b 
depicts the dependence of the resistance for positive (black dots) and negative (red 
squares) bias on the radial position. Therefore, the resistances of up to ten devices 
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were averaged for each position. Defective devices (short-circuited or not properly 
connected to the wiring) and test structures [gray squares in Fig. 7a] were not used 
for averaging. In the inner area of the wafer with a radius of about 25 mm, current 
rectification is observed, and the devices show memristive behavior as described in 
Sect. 2. However, not only the resistance but also the switching window varies with 
position, as can be seen by comparing Fig. 7c with d. The yellow ring in Fig. 7a, 
starting at about 30 mm away from the center, contains devices with relatively high 
resistance. The Z-V characteristics differ significantly from the functional devices in 
the center [Fig. 7e]. No rectification and no hysteresis can be observed. Here, the race 
track of the target is located. In the outer red area in Fig. 7a, the resistance is rather 
low and comparable to the resistance in the inner area. However, no rectification and 
no hysteresis are apparent [Fig. 7f]. It should be noted that Z-V curves with lower 
voltages are depicted in Fig. 7e, f because the devices suffer dielectric breakdown 
for higher voltages at these locations. 

As described above, devices located at different positions on a 100 mm wafer 
show significant different J-V characteristics even though they were produced in 
one and the same process. Thus, the DBMDs are well suited to correlate plasma 
process parameters with material properties and, finally, with electrical properties 
[49]. The overall goal is first to understand the impact of several plasma parameters 
on the electrical properties and second to use these findings to develop a process to 
fabricate plasma-engineered devices tailored for specific applications. Thus, plasma 
parameters were recorded with a probe consisting of a sensor copper plate (11 mm in 
diameter) connected at the backside to a thermocouple for temperature measurement 
(40 V/K) and a copper bias wire [109]. This was used as a passive thermal probe 
(PTP) [109, 112] and as a Langmuir probe (LP) [113, 114]. A PTP is used to measure 
the energy balance at the location of the sample. The energy flux is dominated by 
impinging particles, surface reactions, incoming radiation, and loss processes such 
as heat conduction and convection through the surrounding gas and by emitted radi- 
ation [106]. Therefore, the energy balance depends on the process parameters named 
above, and it has a crucial impact on film properties [106, 107, 115]. The energy 
balance during Nb, Oy, deposition at different positions is shown in Fig. 8a. With an 
LP, on the other hand, additional process parameters like the electron temperature 
T, [Fig. 8b], the floating potential of the probe ® s; [Fig. 8c], the plasma potential 
P [Fig. 8d], and the current density of positively charged ions to the probe jion 
[Fig. 8e] are measured by applying a voltage sweep to the bias wire and evaluating 
the measured current [116, 117]. While the energy flux and jion show a Gaussian 
distribution with maxima in the center as expected for an unbalanced magnetron, the 
other parameters show a more complex dependency on the position. In particular, 
T, and ®,; have local maxima at the location of the race track, while ® s; has local 
minima approx. 20 mm away from the center, where the devices show the largest 
switching window and the lowest resistance for positive bias together with a pro- 
nounced rectification [compare Fig. 7a—d]. Thus, it is expected that the effect of the 
energy balance superimposes with the other plasma parameters and with the distri- 
bution of negatively charged oxygen ions to obtain a Nb,Oy film having properties 
(e.g., oxidation state and defect concentrations) as needed for memristive switch- 
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of AIP Publishing 


ing in DBMDs. This knowledge can potentially be exploited for engineering device 
performance by using multiple frequency capacitively coupled plasmas (MFCCPs) 
for sputter deposition. An MFCCP allows to control the ion energy and the ion flux 
independently of one another during sputtering and thus can help to control different 
process parameters [118, 119]. 

To show the influence of the plasma parameters on the stoichiometry of Nb, Oy, 
TEM/EELS measurements were performed on two different devices, one located at 
a radial position of 5 mm (showing memristive behavior) and one at a radial position 
of —30 mm (showing relatively high resistance and no switching), respectively [49]. 
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While Nb, O, of the functional device is in a high oxidation state between Nb2O5 
and NbO,, as already shown before [69], the Nb,O, from the non-functional device 
shows significantly less incorporated oxygen and cannot be assigned to one of the 
known niobium oxidation states (i.e., Nb205, NbO2, or NbO). Furthermore, the film 
thickness does vary only by 0.1 nm between both positions and thus within the stan- 
dard deviation of the TEM measurement. The deviation in film thickness does not 
explain the substantial deviation in Z-V characteristics since devices with both thick- 
nesses lead to memristive switching devices if they are located within the center of the 
wafer. Thus, the deviations of the Z-V characteristics can be mainly attributed to the 
chemical and electronic properties of the niobia, while the geometrical properties are 
less important. This can indeed be correlated to the plasma process parameters. More- 
over, the kinetic Monte Carlo simulations [67] introduced in Sect. 2 were extended to 
show the impact of the concentration of the mobile charged defects on the switching 
window [49]. The results are depicted in Fig. 9a. In particular, it is shown that an 
increase in defect concentration ng from 2 - 10?°cm~? to 8 - 10cm? increases the 
switching window at 0.5 V from a factor of approx. 3 to a factor of approx. 4 - 10°. 
Thus, increasing the defect concentration by a factor of 4 does lead to an increased 
switching window by five orders of magnitude. Increasing ion concentration in the 
interface-based HfO,/TiO, device also leads to a larger switching window shown 
by simulations [44]. Furthermore, as it is shown in Fig. 9b, also the voltage range 
in which a hysteresis loop can be observed changes with ion concentration, and the 
reverse current is also more affected as in the model of the DBMDs. 


4.2 Engineering of Filamentary-Based Devices 


This section presents methodologies to engineer the characteristics of the filamentary- 
type memristive devices, such as electro-forming, I-V non-linearity, retention, and 
variability. In particular, engineering methods for the TiO,/HfO, memristive devices, 
that are presented in Sect. 3, are further described. 

The concentration of oxygen vacancies in the metal oxide is one of the decisive 
parameters for the filamentary-type switching behaviors [43]. In the TiO,/HfO, mem- 
ristive device, the HfO, showed a substoichiometric property of x = 1.8 in X-ray 
photoelectron spectroscopy (XPS) measurements. The HfO, layer had an optimal 
environment for efficient electro-forming and filamentary-type switching [43]. In 
the TiO,/HfO, devices, oxygen deficiency in the HfO, was achieved by deliberately 
growing a substoichiometric oxide film using a wedge sputtering method. Another 
approach to modifying the oxygen vacancies in the oxide layer is inserting an oxy- 
gen scavenging metal layer [120]. As already stated above, adjusting the vacancy 
concentration is a common way to adjust electrical switching parameters like form- 
ing voltage or switching voltages [45, 48, 98, 99]. Another method for scaling the 
forming voltage is tuning the thickness of the metal oxide layer. In Fig. 10, the electro- 
forming voltage in the TiO,/HfO, device showed a dependency on the thickness of 
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Fig. 9 Modeled hysteresis 
curves dependent on the 
mobile defect concentration. 
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the HfO,. The thicker the switching oxide layer, the higher the voltage required to 
initiate resistive switching. 

Besides impacting forming and switching voltages, the vacancy concentration 
also affects the switching hysteresis. In Fig. 11a, the simulation result shows such 
an impact on the TiO,/HfO, device. There is an optimum concentration of oxygen 
vacancies at 2.5 - 107° m~3, where the TiO, /HfO, device has the maximum switching 
window at a read-out voltage of —0.1 V, shown in Fig. 11b. The simulation shows 
that the device suffers from a higher concentration of oxygen vacancies due to the 
threshold value of oxygen diffusion [44]. 

Formation and dissolution of filaments are stochastic processes, which means 
the filamentary-type devices typically show a high variability during the switching. 
However, scaling the active area size down can reduce the variability [120]. Moreover, 
the study of the TiO,/HfO, filamentary devices shows that the variability can be 
influenced by the correlation between the thickness of the oxide layer and the active 
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area size. As shown in Fig. 12, an optimal combination of the oxide thickness and 
the active area size exists to obtain reliable switching operation. 

The comparison of the retention time of interface-based and filamentary-type 
TiO,/HfO, devices [see Fig. 4d] implies that the modification of oxygen deficiency 
in the HfO, layer can control the retention property. In this respect, many studies 
show methods to enhance the retention time by introducing retention stabilization 
layers to modify the oxygen deficiency [54, 121]. 
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Abstract The idea of resistive switching devices is originally based on the fact that 
the application of electric fields changes the atomic structure locally and thus also the 
electronic structure of the material. This leads globally to a sustained change in the 
resistance of the material layer, which is generally referred to as resistive switching. 
In resistive switching devices, these atomic reconfigurations are reversible and allow 
the state to be maintained for a long time, which is why the devices are referred to 
as memristive devices (also named Memristor). Memristive devices can be realized 
as two terminal devices in a metal-insulator-metal structure. In the MemFlash cell, 
there is no atomic rearrangement in the device and therefore is a purely electronic 
based switching device. The basic components of the MemFlash cell are floating 
gate transistors, which are reduced from a three-terminal to a two-terminal device by 
means of a diode-like wiring scheme and thus exhibit memristive switching behavior. 
In this book chapter, the MemFlash cell is introduced. 


Keywords Neuromorphic computing - Memristive devices - Floating gate 
transistors 


1 Introduction 


Memristive devices (also called memristors) are mostly based on the effect of resistive 
switching. In resistive switching, the application of an electric field causes a local 
change in the atomic bonding structure of the crystalline atoms of the memristive 
layer. Resistive switching is caused by different mechanism, ranging from magnetic 
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effects through electrostatic effects to atomic configurations. A good overview is 
given in [1]. Based on these effects, a number of device structures for memristive 
devices have been developed [2] and the research on resistive switching devices goes 
back to [3, 4]. 

Even though research on resistive switching materials goes back decades [3, 4], 
interest in the subject has been renewed in recent years. In this context, memristive 
switching materials for non-volatile memory architectures [5] and especially for 
neuromorphic systems [6] are considered as promising candidates. However, both of 
these applications require technology to manufacture memristive devices on a large 
scale, as well as integration or interconnection with the prevailing silicon technology 
[7]. The latter puts research into CMOS-compatible memristive materials and their 
CMOS integration at the center of research interest. This goes hand in hand with the 
development of CMOS integrated memristive devices [7, 8]. 

In this context, a MemFlash cell offer an interesting alternative to conventional 
memristive devices [9, 10], because it is a modified version of a floating gate tran- 
sistor (FGT), which are CMOS integrable devices and commercially available. In a 
MemfFlash cell resistive switching is caused exclusively by a pure electronic effect 
and not based on an atomic reconstruction. 

In this chapter we will introduce the MemFlash cell. First, the concept of the 
Mem Flash cell will be explained and it will be shown how to create a memristive 
device from a FGT. Subsequently, in chapter “Redox-Based Bi-Layer Metal Oxide 
Mennristive Devices”, aspects of the MemFlash cell that are technological important 
will be discussed including an overview of different realizations and extensions of 
the basic concept. Thereafter, the range of applications for neuromorphic computing 
is discussed in this chapter and important properties and conditions for the cell design 
are presented. 


2 The MemFlash Concept 


Mennristive devices are two-terminal devices, which in their simplest structure can 
be realized as a metal-insulator-metal sandwich structure. In a memristive device 
the resistance depends on the charge flow through in the past [11]. It was concluded 
that this is a fundamental property that classifies memristive devices, along with the 
resistor, the inductor and the capacitor, to one a fundamental device of electronics 
[11]. In this context, the reference to the Memristor postulated by Leo Chua already 
in 1971 could be drawn [12]. Even if this conclusion is critically questioned [13, 
14], memristive devices or memristors show fundamental properties which cannot 
be reproduced with one of the other three circuit elements: common with the capac- 
itor and the inductor is the possibility of an information storage (memory effect). 
However, common with the resistor, memristors are passive devices in which no 
energy storage takes place. The memristor is therefore, as the name suggests, a 
memory-resistor. 
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Mathematically, memristive devices can be formally represented by the following 
set of equations: 


I= G(x, V,T)- V with “ = f(x, V,t), (1) 
here x is called a state variable, which is for example a measure of the change in 
atomic configuration within the resistive switching material. The function f(x, V, t), 
describes the voltage driven changes in the atomic structure [11, 15] and is depending 
on the particular memristive system. The conductance of the device G(x, V, t) is 
named as Memductance and a simple relationship can be made between G and the 
state variable x via the relationship [11]: 


G =x: Gon + (1 — x): Goff, 2) 


where Gon and Gopr are the maximum and minimum conductance of the device, 
respectively. Please note, that the state variable x varies between 0 and 1. However, 
the MemFlash cell is not based on atomic reconfigurations but is relies on the FGT. 
An FGT is purely function on electron charge transfer. In this context, it will be 
shown in the following how this device can be converted into a memristive device. 


2.1 Functional Principle of a MemFlash Cell 


A schematic of a simple memory transistor is shown in Fig. la. Basically, it consists 
of a MOS field-effect transistor (MOSFET) with a MOS structure (Metal Oxide 
Semiconductor) in whose gate stack. This MOS stack gives a memory capacitance 
as a gate which controls the channel of the MOSFET and is called a floating gate 
(FG) [16]. Depending on the charge on the FG, the vertical charge flow from source 
to drain through the transistor is influenced. For example, if we consider an n-MOS 
transistor as shown in Fig. la, the drain and source terminals are both heavily n- 
doped while the channel region is p-doped (die Bulk region könnte in der Abbildung 
mit p-Si versehen werden). Depending on the voltage difference between the gate 
electrode and the bulk junction, electrons are accumulated at the interface to the 
gate oxide, resulting in a conductive connection (channel) between the drain and the 
source junction. If the floating gate is sufficiently well isolated from its environment 
and the potential on the external accessible control gate is kept constant, the charge 
state of the floating gate defines the memory state of the transistor. However, floating 
gate transistors are not memristors at first glance, since memristors are two-terminal 
devices while the transistor essentially has three terminals—source, drain and the 
externally addressable control gate (cf. Fig. la). At this respect, the core idea of a 
Mem Flash cell is an external circuit that allows the device to become externally a 
two-terminal device. A circuit that allows this is shown in Fig. la as well. Here, the 
control gate (CG) and source are combined and connected to a common potential, 
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Fig. 1 The concept a MemFlash cell: wiring scheme and I-V curve. a Schematic drawing of a 
MemFlash cell and the diode-like circuitry that turns a FGT into a MemFlash cell (see bold black 
connection between source and gate). b Typical current-voltage curve of a MemFlash cell through 
the circuit shown in (a). The pinched hysteresis is a fingerprint of a memristor 


i.e. the ground potential. Applying a bipolar voltage signal Vps (see Fig. 1) the 
FG is charged or discharged in dependency of the bias direction via the tunneling 
window with the thickness d; and leads to a pinched hysteresis for Ips vs. Vps [9]. 
In other words, the charging or discharging of FG changes immediately the channel 
current Ips. 

As shown in Fig. 1b, a pinched hysteresis loop can be realized with this connection 
as a fingerprint of a memristor. The current-voltage characteristic shown in Fig. 1b 
reflects the curve measured in Ref. [9] and describes the dependence of the channel 
current (Ips) when a bipolar voltage sweep is applied to the drain contact (Vps). 
Here fore, the voltage was increased linearly from zero and then decreased linearly 
to a negative voltage before returning to the origin (see arrows in Fig. 1b). The result 
is a hysteresis. In order to understand this memristive behavior more precisely, a 
capacitative model is developed and described in the following section. 


2.2 Physical Device Model of a MemFlash Cell 


The model used to describe the MemFlash cell is sketched in Fig. 2a and is taken from 
Refs. [9, 10]. In this capacitive model, the potential of the floating gate is expressed 
via 


Orc 


CT 


Vrg = + kcVc +kpVp + kg Vg (3) 


Here, Q rg is the charge stored on the floating gate, and Vc, Vp, Vs, and Vg 
are the potentials of the control gate, drain, source, and bulk terminal, respectively. 
Furthermore, kc, kp, ks, and kg are the respective coupling constants to the floating 
gate which are defined by k; = C;/Cr (i = C, D, S, B). Here, Cc, Cp, Cs, and Cg 
are the respective capacitances and Cy is the total capacitance given by Cr = Cc + 
Cp + Cs + Cpg. In the memristive mode, as already mentioned, a three-terminal 
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floating gate transistor is reconfigured to a two-terminal cell. Therefore, the control 
gate and source potentials are grounded. If we also neglect coupling to the bulk, the 
floating gate potential equation reduces to the following form: 


+kpVp (4) 
CT 


Thus, the floating gate potential is determined by the drain voltage Vp and by 
the charge on the floating gate Q pg, which in turn is determined by Vp and can be 
calculated by 


Org = Orc(to) +f L(Vp, Vrg)dt (5) 


to 


Here, I (Vp, Vrg) is the current that charges or discharges the Fg through the 
tunnel window (the tunnel window is marked by a capacitance Ct in circuit model 
in Fig. 2a). In general, there are different contributions to that current, but the main 
current contributions come from the Fowler—Nordheim tunnel effect and from the 
injection of hot charge carriers [10]. These two effects are outlined in Fig. 2b. In the 
case of Fowler—Nordheim tunneling, the application of a high electric field effectively 
thins the thickness of the tunnel oxide, leading to the tunneling of electrons. While 
hot charge carrier injection is a thermally generated charge transport mechanism in 
which charge carriers can overcome the potential barrier between drain and floating 
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Fig. 2 Physical model of a MemFlash device. a Equivalent circuit model of a FGT. b Different 
charge and discharge mechanisms for the floating gate. Current-voltage characteristics of a Mem- 
Flash cell (taken from Ref. [10]. c The solid line shows the model, while the dots correspond to a 
real measurement. d Floating gate potential curve when a triangular voltage is applied to the drain 
(taken from Refs. [9, 10]) 
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gate at high voltages. Both mechanisms can be described within analytical equations 
and are represented in the model shown in Fig. 2 over two current sources [10]. Here, 
the Fowler—Nordheim current is given by 


Irn = EAjox j AFN f (Erox)? exp(—Brn/Etox) (6) 


where A;,, is the tunneling oxide area, and Ary and Bry are the Fowler—-Nordheim 
tunneling constants. E,,, is the electrical field during programming and erasing, 
Etox = (Vp — Vra)/drox, and depends on the thickness of the tunneling gate oxide 
d,. According to Ref. [10] the equation for the hot carrier injection current can be 
approximated by 


Binj 4 
(cing + Veg)? 


Iinj = EAr £ Ainj exp( Dinj i Vp) (7) 


where Ajnj, Binj, Cinj and Djnj are positive fit constants, which have to be estimated 
from experimental data. In order to describe the current-voltage characteristics with 
the equations, a theoretical description of the MOSFET is still missing. This can best 
be described by the following set of equations: 


y2 
5B [ve — Vih)Vp — > |: 1 +AVp) for Vg > Vin and Vp < VG — Vin 


In = 
2 5 B(VG = Vin)” -(1+AVp) for VG > Vin and Vp > Vg — Vin 
0 for VG < Vih 
(8) 


Here V,;,, 6 and A are the threshold voltage, transconductance and channel length 
modulation parameter of the MOS transistor, respectively. Vg is the gate poten- 
tial, which is equal to Vp — Vrg for Vp < 0 and Vg = Vrg for Vp > 0. Figure 2c 
shows the current-voltage characteristic calculated with the model by using a lin- 
ear triangular voltage sweep and the following set of parameters: à = 0.0625 V~', 
B =28.3 pSV! and V,, = 1.052 V. In addition, Fig. 2c compares the model with a 
typical experimental I-V curve. 

Due to the good agreement between modeled and measured I-V characteristics, 
the physical function mechanism can be well described via the model: for positive 
drain voltage, the floating gate charge is reduced, while for negative voltages, the 
QFG is charged. This changes the channel resistance from the high resistance state 
(HRS) to the low resistance state (LRS) for positive drain voltages and vice versa 
from LRS to HRS for negative voltages. The associated change in floating gate 
potential is shown in Fig. 2d and demonstrates that by changing the voltage at the 
drain terminal, the floating gate potential can be varied. This forms the kernel of the 
MemFilash cell’s memristive behavior and thus the charge on the floating gate can be 
identified as the memristive state variable x, where the following relationship holds: 
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dx dQrc¢ 
—= ,V,t 
dt fe = dt 


= 1,(Grg, Vp, Vra) (9) 


Furthermore, the dynamic function f in the case of the MemFlash cell can be repre- 
sented by the charge current of the floating gate. This allows the memristive behavior 
to be adjusted directly via the charge transport mechanism of the floating gate and 
thus to tailor the cell for its application. The latter will be described in more detail 
below. 


3 Technology and Scaling Issues of the MemFlash 


A major advantage of MemFlash cells over most of the conventional Memristors is 
their compatibility with CMOS technology. The floating gate transistor described in 
the previous chapters was manufactured in a NMOS technology and is thus already 
present in an established technology [17]. This enables the production of large num- 
bers of memristive cells with low variability and high yield. This is crucial for the 
application of Memristors in neuromorphic computing and remains one of the major 
challenges of conventional Memristor technologies. However, as shown in the last 
paragraph, in the MemFlash cell the memristive behavior is mainly described by 
the tunnel current for charging and discharging the floating gate. This places a spe- 
cial technological emphasis on the tunnel oxide and the associated requirements in 
terms of storage duration and transport mechanisms. This defines the technological 
parameters for the current-voltage characteristics of the MemFlash cell and will be 
discussed in the following section. In addition, it will be shown that the concept of 
the MemFlash cell is not limited to FGTs, but can be applied to different types of 
thin film junction-less memory transistors in general. This offers a wide range of 
technological possibilities to tailor the behavior of the MemFlash cell for particular 
application. 


3.1 Tunneling Oxide Scaling 


Memory transistors from the last section, in the form of the EEPROM (electri- 
cally erasable programmable read-only memory) cell, form the basis of today’s non- 
volatile storage media such as flash memory. These memory transistors are manufac- 
tured using the floating gate tunneling oxide (FLOTOX) technology shown in Fig. 3a 
[16]. Within this technology, the gate oxide is thinned at the drain connection so that 
electrons can pass through the tunneling oxide via the transport processes described 
in the last section. This changes the charge on the floating gate, but depends on a 
number of factors, including programming time (the longer the time, the greater the 
change), temperature (the higher the temperature, the fewer hot electrons are avail- 
able for injection), tunneling oxide quality and imposes stringent conditions on the 
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Fig. 3 Scaling of tunneling oxide layer: a the thickness of the gate oxide d; has a significant 
influence on the voltage required and thus on the power consumption. b Dependence of Vp and 
the maximum change in resistance Roff/Ron on the thickness of the tunnel oxide. ¢ Simplified 
schematic representation of the effective tunneling regime via the Fowler—Nordheim tunneling 
process. No charge or discharge was assumed for tunnel currents smaller than 10~! A. An electric 
field strength of 12 MVcm~! was used for the breakdown condition. d Retention of the memory 
cell. Shown here via the change in the floating gate potential as a function of the layer thickness. 
(Figures are taken from Ref. [10]) 


programming voltage required. In particular, a higher voltage amplitude and larger 
programming times are required compared to Memristors, whose resistor switching 
mechanism is based on atomic reconfiguration. Thus, the technological factor that 
plays the leading role here is the tunnel oxide thickness, which is chosen in such a 
way that the memory cell allows information storage for more than 10 years [10, 17]. 
However, in a number of applications, such as neuromorphic computing, short stor- 
age times are acceptable or even desirable and the required voltage of the MemFlash 
cell can be reduced accordingly by decreasing the layer thickness [10]. 

An estimation of the drain voltage for different film thicknesses was made in 
Ref. [10] and the main results are shown in Fig.3. Obviously, by reducing the film 
thickness, the required voltage can be significantly reduced to below 10 V (see black 
data points in Fig. 3b), while still maintaining relatively strong resistance changes 
as depicted as Ror / Ron in Fig. 3b (see red data points). Furthermore, the maximum 
current is also significantly reduced (see inset in Fig. 3b), which leads to a significant 
reduction in the power consumption of the cell [10]. 

In addition, the effective intervals for a charge transport between drain and FG 
were estimated in Ref. [10]. In this case, the voltage interval is limited by the break- 
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down field strength of SiO. to large voltages, while the use of a too low voltage 
leads to the fact that no change can be achieved at the floating gate. As shown in 
Fig. 3c, this interval becomes smaller for quite thin oxide thicknesses. It has been 
found that oxide thicknesses in the range of 4-5 nm lead to a required voltage at 
the drain between 4 and 5 V for FGTs [10]. However, these values lead to quite 
short storage times, as shown in Fig. 3d, and thus rule out a broad application of the 
MemFlash cell in this form. Nonetheless, the concept of the MemFlash cell can be 
applied to different types of memory transistors. In the following section, this issue 
will be discussed in more detail. 


3.2 Different Types of MemFlash 


Figure4 gives an overview of different MemFlash cell designs. Besides the Mem- 
Flash cell which is based on a floating gate transistor (labeled as FLOTOX in 
Fig.4), SONOS (silicon oxide nitride oxide polysilicon) transistors [18], quantum 
dot devices [19] and also memory transistors with a gate which splits into a y geom- 
etry [20] have been realized so far. In the following section, these other types of 
MemFilash cells will be briefly discussed. 

In Ref. [18], the MemFlash concept was applied to industrially manufactured 
SONOS field-effect transistors. To account for the special characteristics of depletion- 
type SONOS transistors, the two-terminal wiring of the MemFlash cell was extended 
to include a resistance-bridged pn diode. Common SONOS devices have very thin 
layers of the ONO gate stack, with an approximate thickness of 2 nm for the tunnel 
oxide (O), 5—10 nm for the nitride layer (N), and 5 nm for the barrier oxide (O) toward 


FLOTOX SONOS 


Fig. 4 Different memory transistors used as MemFlash device: these include floating-gate oxide 
(FLOTOX) transistors [9], silicon oxide nitride oxide polysilicon (SONOS) field-effect transistors 
[18], transistors with a Y-gate [19], and quantum dot (QD) floating gate transistors [20] 


124 H. Winterfeld et al. 


the gate electrode. Thus, this type of MemFlash cells promise reliable operation, low 
parameter spread and high integration density [18]. 

A completely different approach was taken in Ref. [19]. Here a MemFlash cell 
was realized with a quantum dot (QD) floating gate transistor by shorting the source 
contact of the quantum wire (QW) and lateral gates. Here, the memristive behavior 
arises from the Coulomb interaction of localized QD charges with the nearby QW. 
This leads to a change in conductance that depends on the voltage polarity. 

Finally, an energy-efficient memristive floating-gate device was presented in Ref. 
[20]. This device was fabricated in a standard 180 nm CMOS process and connected 
to a readout transistor. This concept, known as a Y-flash cell, allows the device 
to operate in an energy-efficient subthreshold memristive mode. This allows small 
signal changes to be linearized, so that a dynamic resistance range of two orders of 
magnitude can be achieved. This allows 65 discrete resistance levels to maintain long 
data retention, high endurance and a low noise margin. 

An extension of a memory transistor by another transistor was proposed in Ref. 
[21]. The idea of this approach is to combine a MemFlash based on a complementary 
n-MOS transistor/p-MOS transistor arrangement, similar to conventional CMOS 
inverters (not shown in Fig. 4). This concept, called C-MemFlash, allows the voltage 
polarity and the direction of the resistor switching to be adjusted by means of an 
appropriate wiring scheme. 


4 MemFlash Cells for Neuromorphic Computing 


One of the increasingly important areas of application for Memristors is neuromor- 
phic computing [22]. Here, Memristors are used to emulate synaptic plasticity, i.e. 
the changes in coupling strengths between neurons. In neuronal networks, synapses 
play a special role, since these variable changes enable local and decentralized infor- 
mation processing. Thus, synaptic plasticity is a basic building block for learning 
and memory processes in neuronal networks and a detailed modeling is required 
to achieve functional neuronal networks [23]. A large number of different models 
that mimics neuronal learning are known, whereby the theory of Hebbian learning 
models allows a quite extensive description of these [24]. In the following, the pos- 
sible applications of MemFlash cells in the field of neuromorphic computing will be 
presented in the context of this model. 


4.1 Hebbian Learning Models 


Hebbian learning models goes back to a rule postulated by Donald Hebb in 1949 
[25] which states that “neurons that fire together wire together.” This means that 
neurons that are simultaneously active change their connection strength also named 
as synaptic weight. This learning rule can be easily translated into a mathematical 


MemFlash—Floating Gate Transistors as Memristors 125 


framework by denoting the change in the coupling strength is defined by a function 
which describes the activities A; or A; of the adjacent neurons, i.e. the pre- and 
post-synaptic neurons, and the actual weight of the synapse @;; [26]: 


dai; 
dt 


In the simplest case the function F (wij, Aj, Aj) can be written as: 
F(%ij, Aj, Ai) = aA; - Aj (11) 


Here, the coefficient a is called learning rate and is usually positive: a > 0. A 
learning rule with œ < 0 is usually called anti-Hebbian [26]. However, the exciting 
question is how this learning theory can be applied to the MemFlash cell. Here, a 
structural similarity between the equation for the change of the synaptic weight w;j 
and the change of the memristive state x can directly be recognized, which allows 
us to derive the following relation [27]: 


dw dx dQrcG 
> > 
dt dt dt 


= 1,(QFc, Vp, Vra) (12) 


Thus, the change in synaptic coupling strength can be directly modeled by the 
change in floating-gate charge [24, 27]. 

Furthermore, the learning function F (@;;, Aj, A;) is simulated via the tunnel cur- 
rent and the pre- and postsynaptic activities can be identified as the potentials at the 
drain and floating gate. For this purpose, voltage pulse trains at the drain terminal 
can be used, as sketched in Fig.5a, which must be adjusted in their voltage level, 
pulse length and frequency to emulate the desired learning behavior. An example 
that emulates the change of synaptic weight via the conductance of the MemFlash 
cell is shown in Fig. 5b. Here, constant frequency pulse trains of 175 10 V set pulses 
(also called potentiation pulses) and 175 —7.5 V reset pulses (also called depression 
pulses) were used, which were varied in their pulse width. A read pulse of +2.5 V was 
consecutively applied after each potentiation/depression pulse to non-destructively 
readout the channel conductance. Figure 5c shows the normalized synaptic weight 
changes w(t) (floating-gate charges) for the different pulse widths. Here, a strong 
nonlinearity of the weight change in the respective first potentiation and depression 
pulses can be observed. An exception is the 5 ms curve, which shows no unlearning 
behavior. This result can also be seen from the curves shown in Fig. 5d. Good agree- 
ment with a variety of biological plasticity processes has been discussed in Ref. [24]. 
The application of cellular learning paradigms in the area of associative learning 
was also demonstrated in [24]. In addition, a broad range of applications of Y-gate 
MemFlash cells for neuromorphic architectures was presented in Ref. [20]. 
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Fig. 5 Application of MemFlash cell in neuromorphic computing: a replication of synaptic plas- 
ticity as a change in FG charge. For this purpose, the floating gate charge can be adjusted by suitable 
pulse trains at the drain potential and thus the conductance of the MemFlash cell. b an example of 
an emulated plasticity curve from Ref. [24]. This shows a strong non-linear behavior in the change 
of conductance (c). d The learning rates are thus initially very high and become lower with the 
number of pulses. (Figure b—d are taken from [24]) 


5 Conclusion 


In summary, it can be stated that by means of defined charge changes of the floating 
gate, controlled learning models can be realized in hardware. In addition, the existing 
technology integration of MemFlash cells with the associated good reproducibility 
and high production capability of a large number of cells offers a good possibility 
for the realization of neuromorphic computing architectures in hardware. 
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Abstract Memristors are promising candidates for new memory technologies and 
are capable to mimic synapses in artificial neural networks. The switching in mem- 
ristive devices occurs typically in few nanometer thin dielectric layers. The direct 
observation of the switching mechanism is crucial for better comprehension and 
improvements of memristors. Therefore, in situ experiments are conducted in a 
transmission electron microscope (TEM). However, sample preparation processes 
and electron beam irradiation can lead to a chemical and structural modification of 
the active layers. Moreover, devices may show significant device-to-device variabil- 
ity due to the details of processing parameters. Thus, it is essential to characterize 
the identical device electrically before microstructural analysis. 
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1 Introduction 


Is Moore’s Law at an end? Memrisitve devices can help to extend the functionality 
of today’s state-of-the art digital computers while simultaneously decrease energy 
consumption for data intensive and associative tasks in novel in-memory and bio- 
inspired computing architectures [1, 2]. The usual two-terminal device structure can 
further help to decrease the size of integrated circuits and memory arrays. 

Simple two-terminal memristive devices can help to further decrease the feature 
size in integrated circuits, while simultaneously decreasing energy consumption. 
However, with decreasing feature size, every single structural and electronic defect 
becomes more important and analytical techniques become the limiting factor for 
knowledge-driven engineering of memristive devices. In this chapter, the limits of 
TEM are critically discussed. TEM, as one of the suitable analytical methods, pro- 
vides sufficient resolution as well as a plethora of different techniques to obtain 
information from real space, reciprocal space and spectroscopy. Recent progress in 
TEM analysis of memristive devices is reviewed in publications of Strobel et al. [3], 
Yang et al. [4], Sun et al. [5], Xu et al. [6] and Yang et al. [7]. 

The final goal of most TEM studies on memristive devices is to identify the 
switching mechanism by directly observing structural or chemical changes or inho- 
mogeneities inside the memristive material. However, the identification of finger- 
prints for memristive switching is not straightforward as discussed below. 

The most studied switching mechanism is the growth and rupture of a conduct- 
ing filament which is believed to be essential for the electrochemical metalization 
mechanism (ECM) and in valence change memories (VCMs) [8]. In memristive 
devices based on the ECM, a metallic filament forms due to the partial dissolution 
of an active electrode (typically Ag or Cu) into a dielectric layer in high electric 
fields. Such metallic filaments were often observed in situ in TEM experiments, as 
summarized in the review from Yang et al. [4]. However, radiolysis and developing 
electric fields in the sample due to the electron beam complicate such experiments 
and can lead to unambiguous interpretations. Especially for the thin TEM samples, 
surface-migration and surface oxidation should be prevented to reduce effects from 
radiolysis and electric fields. 

One of the main defects discussed for resistive switching in the field of VCMs are 
oxygen vacancies, which are reported to be responsible for the formation of one or 
multiple conducting filaments inside the dielectric matrix [9]. These vacancies, how- 
ever, are generally difficult to detect. In the case of TiO} memristors, such filaments 
could be identified by small crystallized regions with decreased oxygen content in 
the so called Magnéli phases [10]. These can be directly identified by diffraction and 
imaging techniques in the TEM. In other oxides, however, such direct fingerprints 
for the filaments are under debate and thus indications are used to identify the fil- 
aments, e.g., by spectroscopic means. Li et al. [11] and Jang et al. [12] observed 
such an indication for HfO2 in the low loss electron energy loss spectrum (EELS): 
A plasmon peak change under in situ biasing was interpreted as the change of oxy- 
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Fig. 1 a Low-loss EEL spectra with bias voltages from 0 to 10 V. The slit energy window for the 
energy filtered (EF)TEM images in b is indicated by orange dashed lines. The color code indicates 
higher oxygen vacancy concentrations. Adapted by permission from [Springer Nature Customer 
Service Centre GmbH]: [Springer] [Nano Research] [11] 


Intensity (a. 


Intensity (a.u.) 


Fig. 2 Annular bright field STEM image of a Au/Fe:SrTiO3/Nb:SrTi03 memory device after 
electroforming and delamination of the Au top electrode. Adapted with permission from [ACS 
Publications][Chemistry of Materials] [9] 


gen vacancy concentration and distribution, which could then be mapped in energy 
filtered (EF)TEM images (see Fig. 1b). 

An additional difficulty is presented by the fact, that conducting filaments can be 
formed anywhere in the memristive device, while TEM can only analyze a limited 
region of interest. Yang et al. circumvent this problem by electrically probing a HfO2 
thin film with conductive-AFM and fabricating FIB lamellas from probed regions 
[4]. Such a filament was subsequently observed by means of scanning TEM (STEM) 
energy dispersive X-ray spectroscopy (EDX) elemental mapping, as it exhibited 
a reduced oxygen signal. Du et al. delaminated the top electrode of a device after 
cycling and prepared FIB-lamellas from regions where they detected defects [9]. The 
observed Ti*+ ions indicated oxygen vacancies in SrTiO3 which could be identified by 
X-ray absorption spectroscopy and X-ray photoemission electron microscopy. Con- 
sequently, the researchers were able to show that multiple filaments can be formed 
simultaneously, but with different velocities (see Fig. 2) [9]. Accordingly to this, 
an observed filament can be in any resistive state, since a different filament might 
dominate the resistive switching. Nevertheless, observing and characterizing the fil- 
aments present great opportunities to gain a fundamental understanding of switching 
mechanisms and towards knowledge-based improvements of memristive devices. 
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However, such experimental results impose a high margin of error since artefacts 
could be introduced by TEM sample preparation and by beam damage during mea- 
surements. Accordingly, great effort should be made to rule out or minimize artifacts. 
Therefore, the following chapter presents means to achieve suitably pure samples, as 
well as aspects of best practice. These include TEM sample preparation and the pos- 
sible induction of artifacts, performing TEM measurements without beam damage 
and ultimately dedicated in operando methods. 


2 TEM Sample Preparation 


TEM measurements require a specimen thickness below 100 nm and for reliable 
(electron energy loss) spectroscopic information the thickness should be below 
50 nm. This requirement makes the preparation equally important as the experi- 
ment and the analysis itself, because unwanted alterations of the material or the 
device are to be avoided. When analyzing memristive devices, electrical character- 
ization prior to preparation and structural characterization via TEM is crucial and 
should be carried out on the same device that is to be measured. Memristive devices 
generally exhibit a high variability from device to device, which complicates the 
subsequent analysis. Even devices on the same wafer can differ strongly, depending 
on their position on the wafer. These differences could be attributed to local variances 
in plasma conditions during sputter-deposition, which result in different oxidation 
states of the metal oxide components on the wafer, which was proven by EELS fin- 
gerprinting of the O-K edge in this case of a niobium oxide thin film [13]. Challenges 
of preparations of specific devices are summarized below. 


2.1 FIB Preparation of Specific Devices 


The preparation of cross-sections of single devices on a lithographically structured 
wafer relies on Focused Ion Beam (FIB) milling with Ga ions combined with scanning 
electron microscopy to select a specific region of interest. The FIB method enables 
the preparation of a cross-sectional lamella, a few um in size, with a homogeneous 
thickness for high quality measurements. Prior to the Ga ion milling, a protective 
Pt-layer is deposited with a gas injection source (GIS) that reduces both charging and 
Ga-implantation in the device. In case of insulating specimens, this Pt-layer might 
be insufficient as strong charging hinders the Pt-deposition itself. In case of sapphire 
substrates, electron and Ga ion bombardment can lead to an amorphization of the 
surface (see. Fig. 3). An alternative preparation method would be using the precision 
ion polishing system (PIPS), which is described further below. However, due to the 
method requiring steps of mechanical grinding and thinning steps, preparation proved 
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Fig. 3 a HRTEM micrograph of hafnium zirconium oxide (HZO) grown on sapphire substrate, 
prepared as a FIB cross-section. A high defect density is visible at the surface of the substrate. The 
orange rectangle indicates the magnified region in b 


to be impossible, as sapphire is one of the hardest materials. In the case of memristive 
devices, the top electrode protects the underlying layers and reduces charging effects 
during the FIB preparation. 

Independent from the charging effects in the FIB, (e.g., by a sputtered top elec- 
trode) dedicated TEM measurements require a final thinning step of the FIB lamella. 
This can be done in the FIB with Ga ions accelerated with lower voltage from the 
top of the cross-section. However, this procedure is prone to Ga implantation and 
surface amorphization. 

An alternative approach is to perform the final thinning with Ar ions with low 
voltages to gently sputter material from the FIB lamella. However, for sapphire 
substrates, low-energy milling is required, as it has been observed that sapphire is 
sensitive to Ar ion milling at energies above 3 keV [14]. Sapphire becomes suscep- 
tible to electron beam damage in the TEM, which could be prevented when milling 
specimens with Ar ions with acceleration voltages of 2.5 keV or less [14]. This can 
be explained by the incorporated Ar point defects that act as nucleation centers for 
the reduction of Al,O3 in the electron beam [15]. Accordingly, oxides, especially 
with high defect concentrations, should be Ar-milled only if necessary and with 
low accelerating voltages (e.g., 500 eV) to prevent the creation of defects in the 
preparation. 

Different milling rates along the lamella in the FIB might occur in cases, where the 
top most layer is inhomogeneous or structured with different materials (e.g., with top 
electrodes or dielectrics). In these cases it might be helpful to flip the lamella by 180° 
and to perform the FIB thinning from the backside, i.e., the side of the substrate [16]. 
Particularly for the most common substrate in many fields, a crystalline Si wafer, this 
procedure enhances the homogeneity of the FIB lamella. Still, a slightly misoriented 
Si substrate can result in strong anisotropic etching from different etching rates of 
different crystal orientations, as can be seen in Fig. 4a, while in Fig. 4b homogeneous 


134 O. Gronenberg et al. 


Fig. 4 Two FIB lamellas prepared with a backside thinning procedure, developed in-house, using 
a rotation tip. In a, the lamella was slightly tilted due to stress during transfer to the rotation holder 
for the 180° rotation. As a result, a saw tooth structure developed due to anisotropic etching in the 
crystalline Si substrate. In contrast, the lamella in b was perfectly aligned for backside thinning 
resulting in a smooth Si surface 


milling was achieved. In Fig. 4a, the FIB lamella was lifted out and attached to a 
rotation tip for the 180° rotation. During this transfer, the lamella was slightly tilted 
by stress, leading to a sawtooth structure due to anisotropic etching of the crystalline 
Si substrate. 

In cases where it is suspected that the preparation has produced artifacts, a refer- 
ence sample must be prepared with a well-established and more sensitive preparation 
method. Such a method could be using the PIPS at low voltages, which mills a hole 
inside the sample with Ar ion impact at an oblique angle. With the PIPS, no spe- 
cific devices can be selected as TEM specimens, though high-quality specimens can 
be produced. A comparison between FIB and PIPS preparation, acting as check of 
reliability, has been carried out by the analysis of low-temperature superconducting 
Josephson junctions as a reference system [17]. Figure 5a exemplifies the slightly 
different outcome of these procedures: the sample prepared with FIB shows lower 
contrast and resolution than the sample prepared via PIPS because of higher thick- 
ness and some FIB induced amorphization of the surface layers [18]. Importantly, 
though, the STEM-EDX scan (Fig. 5b) across the layers yielded qualitatively the 
same result for both cases, showcasing the partial oxidation of the Al layer on the 
side facing the surface. Since the results show such congruence, the preparation by 
FIB - which is necessary for the actual devices - is deemed suitable [19]. 
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Fig. 5 Comparison of a Josephson junction prepared via FIB (left) and PIPS (right). In a, HRTEM 
shows the polycrystalline nature of the Nb top and bottom electrodes as well as the partially amor- 
phous Al/AIO, layer in between. The EDX scans in b, performed along the yellow arrows in a, 
show both that Al is partially oxidized on the side facing the surface. Adapted with permission of 
AIP Publishing [19] 


2.2 Conclusion for the Preparation 


Protect the topmost layer before preparation to prevent Ga implantation and related 
artefacts 

Use sensitive preparation methods with low acceleration voltages 

Check for preparation artefacts in the beginning, before starting sophisticated and 
time-consuming measurements 

If preparation artefacts are likely, compare the same or a well-comparable sample 
prepared with a more sensitive method (e.g., PIPS) 

Best practice: if possible fabricate a (prototype) device on a TEM-compatible grid 
to check functionality before, after, or even during measurements. 


3 Spectroscopic Methods 


Modern TEMs are equipped with EDX spectrometers as standard, since characteristic 
X-rays are readily formed when the electron beam passes through the sample. These 
X-rays can be used to determine and even quantify the elements in respect to their 
concentrations in the sample. Not only the primary beam generates X-rays, but also 
secondary electrons that interact inside the microscope column and generate spurious 
X-ray signals. These spurious X-rays, as well as further artifacts and pitfalls, will be 
discussed in the following chapter. 
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Fig. 6 a STEM high angle (a) 

annular dark field (HAADF) 1 0 nm 
image showing the thin film 
stack of a DBMD build up 
from bottom to top of Nb, 
Al203, HfO2 and Au. The 
spectrum image in b is 
created from the Ar EDX 
signal and in c from the Ar 
M23 EELS signal. The inset 
in c shows the EEL spectrum 
with the Al203 plasmon and 
the sharp Ma3 threshold peak 
of Ar at 12 eV. The 
overlayed image in d shows 
the Al EDX signal in orange 
and the Ar EELS map of ¢ 


(b) EDX Ar-map 


Se 


Spurious X-rays are characteristics of the respective microscope and TEM grid 
in use. Reference spectra next to the sample give a clear indication of which signal 
is spurious or intrinsic. This procedure fails when the TEM grid contains elements 
which are also present in the actual specimen. Accordingly, the right TEM grid needs 
to be chosen beforehand. Further limiting this choice are overlapping EDX peaks 
between TEM grid and elements that are expected to be in the sample. For instance, 
peaks of Si and Mo or Ti and Ba overlap and cannot be distinguished with the energy 
resolution of common EDX detectors. 

Artificial peaks can also occur and are produced as sum-peaks when two charac- 
teristic X-rays are detected within the processing time of a signal. These sum-peaks 
can usually be ignored but become critical when an element at the same energy is 
falsely detected. One prominent example is the sum-peak of Al, which lies at the 
same position as Ar [20] and could lead to the wrong exclusion of Ar in Al. In a study 
of a double barrier memristive device (DBMD) including a roughly 7 nm thin Al203 
layer (compare Sect. 4 dealing with this device), an Ar EDX signal was measured 
exactly in the middle of this layer as shown in Fig. 6b. In this case, an Al sum-peak 
could be ruled out by the complementary technique EELS. The Ar Ma; edge at 
12 eV (see inset in Fig. 6c) and the L23 edge at 320 eV prove the presence of Ar in 
the Al,O3 thin film, which is also indicated by the EDX signal. The EEL spectral 
image exhibits a significantly better signal-to-noise ratio at the same acquisition time 
in comparison to the EDX map. The compound map in Fig. 6d of Al (EDX signal) 
and Ar (EELS signal) shows a clear anti-correlation, suggesting Ar segregation in 
less dense regions inside the Al,O3. Dark contrast in the STEM-HAADF image in 
Fig. 6c supports this hypothesis. 
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The usage of complementary techniques can not only support the measurements 
but also point the way to further conclusions. As an example, a combination of 
EDX, X-ray photoelectron spectroscopy (XPS) as well as UV-vis spectroscopy was 
used to investigate a system of bimetallic nanoparticles. These nanoparticles were 
fabricated via a custom-made multi-component (e.g., AgAu) target in a Haberland- 
type gas aggregation cluster source (GAS) and subsequently studied. Since XPS 
and the optical behavior of the sample rely strongly on the surface properties, the 
comparison with EDX can reveal the difference of bulk and surface composition, 
even though the particles are 10 nm in diameter. XPS quantifications showed a 
systematically higher Ag content in comparison to EDX, which led to the assumption 
that an Ag enrichment had occurred on the surface of the AgAu nanoparticles, even 
though the Ag-Au system is perfectly miscible. This hypothesis is supported by 
quantitative approximations of plasmon peak positions in UV-vis spectra shifting 
with the composition (see Fig. 7), which can be controlled with the pressure inside 
the GAS [21]. 


4 Example Characterization of Double Barrier Memristive 
Devices 


In this section, an extensive investigation of double barrier memristive devices 
(DBMDs) is presented, from which knowledge-based improvements could be 
derived. These DBMDs were designed with an active layer (i.e., NbO, or HfO,) 
that is deposited on the bottom part of a Josephson junction (Nb/A1/A1203) [17] 
and a Schottky-type contact to an Au top electrode. This device is believed to show 
interfacial resistive switching which is reported to result from oxygen ion move- 
ment affecting the interfacial properties [22]. Therefore, no fingerprints for resistive 
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Fig. 8 STEM and spectroscopic investigations of the first generation DBMD. The compound map 
from EDX signals in a illustrates the layout of the device. The goodness-of-fit map in b of the O-K 
edge of the NbOy/, shows the surface-oxidation of the Nb bottom electrode and its roughness. The 
O-K edge of AlO, in ¢ contains a pre-peak feature which was fitted with a Gaussian function, while 
the main edge was fitted with a Pearson I(V). Adapted with permission from AIP Publishing [19] 


switching, such as filaments that could be directly observed with TEM, are known 
to characterize these devices. 

These devices were analyzed by spatially-resolved STEM-EELS and -EDX to 
simultaneously map light and heavy elements. ELNES (energy loss near edge struc- 
ture) fingerprinting was utilized to determine the oxidation states of the Nb atoms. 
In this approach, the shape of the edges is compared to known reference spectra. In 
the first-generation DBMD, the ELNES of the alumina resembles that of amorphous 
Al,O3. Unlike the targeted (Nb/AI/A1,03)-Josephson junction, the entire Al layer 
and even the surface of the Nb bottom electrode are oxidized. This niobia (NbO,) 
could be assigned to Nb(II) oxide, thus providing sufficient conductivity for device 
functionality. The Nb bottom electrode and Al layer were sputtered as metallic thin 
films, which subsequently oxidize during the reactive sputtering of the memristive 
layer on top (i.e., NbO,.). However, the ELNES of this NbO, could not be clearly 
assigned to a certain Nb oxide. A mixture of Nb(IV) and Nb(V) appears probable. 
Further, the electronic structure of the AlO, layer measured with EELS contains an 
additional pre-peak feature at the O-K edge which is located roughly 6 eV before 
the main edge (see Fig. 8c). A similar feature, which evolves due to beam damage 
in oxides, is reported in literature - its origin will be discussed in more detail later 
on [23, 24]. In this case, however, beam damage appears unlikely, as no evolution of 
this pre-peak over time and no morphological changes even after extended electron 
irradiation could be observed. Another explanation of this pre-peak feature could 
be oxygen vacancies in AlO, as suggested by Nigo et al. [25]. This feature might 
be caused by localized Al(3s) states in the band gap of AlO,. By comparing the 
area ratio of the pre-peak and of the main edge, the concentration of these vacancies 
could be estimated to be in the range of 1.7 to 3.2% with respect to the entire oxygen 
content [19]. A third possible explanation for this feature is a superposition of the 
O-K edges of alumina and the neighboring niobia, which was observed by Bruley 
et al. [26]. The onset of the O-K edge from niobia lies few electron volts lower than 
that of alumina. This explanation seems quite convincing, as the Nb-M3 and Nb-M» 
edges are present at the same position and the shape and the position of the pre-peak 


Critical Discussion of Ex situ and In situ TEM Measurements on Memristive Devices 139 


resembles the onset of the two neighboring niobia O-K edges. Both interdiffusion as 
well as an imperfectly aligned FIB lamella with respect to the electron beam could 
explain this scenario. 

In addition to spectroscopy, simultaneous morphological analysis can yield further 
valuable information. The roughness of the bottom electrode leads to a minimal 
distance between NbO, and NbO, of around | nm across the AlO, (see Fig. 8, the 
yellow lines), which might enable tunneling through this oxide. In contrast to the 
targeted Nb/AI/Al,03, no metallic Al smooths the interface to the dielectric AlO, 
[27] which should lead to an inhomogeneous electric field between the electrodes. 

According to [19], these devices demonstrated a high robustness regarding varia- 
tions from the targeted design and layer sequence. These DBMDs were still functional 
after NbO, was subsequently replaced by HfO,. 

Nevertheless, only devices within a central area of a 100 mm wafer were functional 
(inner red area in Fig. 9a). To understand the reason for this, plasma and TEM 
characterizations were carried out and revealed distinct differences. In particular, the 
reactive plasma process to deposit NbO, layers with metal targets using an Ar/ Oz gas 
atmosphere leads to variable oxidation states of Nb atoms across a whole wafer [13]. 
The subsequently patterned devices are showing a distinct memristive functionality 
in the center of the wafer, while the devices at the edge of the wafer exhibit no 
memristive I(V) characteristics. These findings indicate a strong correlation between 
local plasma parameters and device performance and may pave the way to plasma- 
engineered memristive components in the future. In case of NbO,., the ELNES of the 
O-K edge varies between different wafer positions. In the functioning device from 
the center of the wafer, the ELNES could be assigned to either Nb(IV) or Nb(V) 
oxide, with slight deviations to both. A mixture of both oxides is probable. However, 
from a non-functional device from the edge of the wafer (see I(V)-characteristics in 
Fig. 9d), the O-K edge could not be assigned to a certain oxidation state. The lower 
signal-to-noise ratio in comparison to the other oxides with the same acquisition 
parameters, however, suggests a lower Leerschritt O-K edge signal and with this a 
lower oxidation state of the Nb atoms. In addition, niobium oxide with a low valence 
state is known to be a conductor and cannot be used to build a Schottky type barrier 
as observed from the I(V) characteristics [13]. In summary, it could be proven that 
the EEL spectroscopy of the niobium oxide O-K edge could explain the different at 
the specific locations on the wafer. 

Independent from the top-most oxide (i.e., NbO, or HfO,), the AlO, layer under- 
neath could be assigned to amorphous Al,O3 by the O-K edge. However, exactly in 
the middle of the AlO, thin film a pre-peak feature appears as shown in Fig. 10a, 
which clearly differed from the pre-peak from previous studies as shown in Fig. 8c. 
Pre-peaks of the O-K edge occupied researchers for a long time, but still, the origin 
and meaning are under debate. The pre-peak can be part of the main O-K edge in 
some minerals, instead of being an additional defect feature, as shown by Winkler et 
al. using DFT calculations, e.g., for Hambergite (Be2BO3(OH)). They have further 
demonstrated that the “pre’’-peak originates not only from the OH-, but also from all 
BO3-groups [28]. An older hypothesis from [29], which claimed that the hydrogen 
content of minerals can be determined from the pre-peak feature, can therefore be 
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Fig. 9 a Resistance map of a whole wafer with memristive devices deposited on top measured at 
1.6 V. b and c HRTEM micrographs from positions comparable with 2 and 3, respectively, indicated 
in a. The STEM-EELS maps of the Nb-M45 edge showing the Nb distribution. d illustrates the 
O-K EEL spectra in the niobiumoxide of the different positions overlaid with reference spectra. The 
I(V)-characteristics of these devices at the respective positions are shown in the insets. Adapted 
with permission from American Vacuum Society [13] 


ruled out. The same conclusion was stated by Garvie, who assigned the pre-peak 
of the O-K edge in OH- and H2O-bearing minerals to beam damage, which could, 
however, be prevented by low dose-rates [30]. The pre-peak appearing as a result 
of beam damage originates from the transition of core electrons to unoccupied pi* 
orbitals of molecular oxygen [23, 31-33], which develops, e.g., due to the decom- 
position of hydroxides [24] or the reduction of the oxide itself [34, 35]. As indicator 
for beam damage serves its development over time, as the damage process occurs 
gradually and the resulting Oz gas is emitted into the vacuum. A more detailed look 
on beam-induced damage is provided in the following Chap. 5. A recent study from 
Ayoola et al. proposed that peroxy O-O bonds developed at the surface of y-Al,O3 
in the areas damaged by electron irradiation. However, these O-O bonds could be 
prevented by employing cryogenic-EELS although beam damage in form of hole 
drilling still occurred [36]. Peroxy groups were also found in amorphous Al,O3 by 
stochastic quenching simulations which best explains a similar pre-peak of the O-K 
edge measured by Near Edge X-ray Absorption Fine Structure [37]. Further, single 
superoxide ions (Oj ) were detected in sapphire with electron paramagnetic reso- 
nance after neutron-irradiation, proving their possible existence in Al,O3 [38]. It 
seems likely though, that peroxy groups in Al203 are unstable in the electron beam 
and directly decompose into molecular oxygen. Fritz et al. reported on an electron 
beam stable pre-peak at the grain boundaries of polycrystalline Al,O3, and it was 
interpreted as chemisorbed molecular oxygen [39]. 

In our own measurements, the pre-peak in the amorphous Al,O3 in the DBMD 
(Fig. 10a) does not emerge or increase over time, but disappears with electron beam 
irradiation (see Fig. 10a), which could be an indicator for adsorbed oxygen molecules 
released to vacuum. This hypothesis is supported by the difference spectrum before 
and after electron irradiation in Fig. 10a, which resembles the O-K edge of molecular 
oxygen gas. The pre-peak does not only disappear in the vicinity of the electron 
beam, but also in distances further than 50 nm, which can be seen in the pre-peak 
spectral image in Fig. 10e with an exposure time of 100 ms. Here, the oxygen 
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Fig. 10 a Time-series of the background-subtracted O-K edge in Al203 with pre-peak feature 
(green) in the beginning and after the pre-peak vanished (red). The difference of these spectra 
(black dashed line), after alignment of the onset of the main O-K edge, resembles the O-K edge of 
oxygen gas. The Al-L23 edges in b recorded simultaneously with a, show no significant difference, 
indicating no valance change of Al203 and no metallic fractions. The STEM-HAADF image in c 
provides an overview for the sequential spectral images of the pre-peak of the O-K edge in d and e. 
d was recorded first with an exposure time of 9.8 ms and e was recorded second with an exposure 
time of 100 ms. In fast measurements the pre-peak appears spread throughout the entire Al203, 
while in slow measurement, the pre-peak concentrates at the left corner where the scan started. The 
yellow dashed line indicates the interface to the Nb bottom electrode and the orange dashed line 
indicates the center of the Al203 where the pre-peak is located. The oxidized Nb bottom electrode 
shows intensity in the oxygen pre-peak spectral image in e, because the onset of the main O-K edge 
of niobium oxide is located at the same energy. In the compound map f, the O-K pre-peak (similar 
to e, but different position) and the Ar EDX signal are anticorrelated. No Ar is detected where the 
O-K pre-peak concentrates but elsewhere Ar does not vanish 


pre-peak concentrates at the left corner, from where the scan was initiated, while 
a fast measurement beforehand (Fig. 10d) proved that the pre-peak spreads over 
the entire Al,O3. This long-range effect could be explained by two scenarios. In 
the first, secondary electrons spreading in all directions from the focused electron 
beam, which causes the pre-peak to vanish before it could be measured in EELS. 
These secondary electrons have lower energy and a shorter mean free path and may 
produce stronger beam damage by radiolysis (for more details see Sect. 5 about beam 
damage). In the second scenario, the focused electron beam positively charges [35] 
the Al203, which could attract negatively charged chemisorbed [40, 41] oxygen-gas 
molecules. 

An argument for the second scenario is that the intensity of the concentrated 
pre-peak appears higher, when comparing the second, slower scan to the initial fast 
scan. This could be explained by the attraction of negatively charged chemisorbed 
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oxygen, not only from the right side of the scanning window in the Al,O3 but also 
from the left. In a time-series (see Fig. 10a), no further increase in the pre-peak was 
observed but only a decrease, which argues against a development over time and 
therefore against a pre-peak produced by beam-damage. In the pre-peak-map in Fig. 
10e, intensity can be observed below the yellow dashed line, roughly indicating the 
interface to the Nb bottom electrode. However, this intensity is produced from the 
main O-K edge of the oxidized Nb, which is located at the same energy and therefore 
mapped as well. In the fast measurement in Fig. 10d the noise level is too high to 
detect the Nb O-K edge. 

The presence of Ar gas inside the Al.O3 further proves the possibility for gas 
inclusions and thus for incorporated oxygen gas. Ar gas was detected at the same 
position as the molecular oxygen in the middle of AlyO3 by EDX and EELS (compare 
Fig. 6). The Ar signal disappears, however, where the O-K pre-peak concentrates, 
even though Ar gas is present where the O-K pre-peak has vanished by the long- 
range effect (see Fig. 10f). This may indicate a physical displacement of the Ar atoms 
from the oxygen molecules, as the Ar signal is stable in the electron beam. However, 
molecular oxygen could have also been formed due to the preparation, where the 
Ar atoms could destabilize the Al,O3 as proposed by Nufer [15]. In this case, the 
molecular oxygen should be the product of the reduction of Al,O3 to metallic Al, 
but no temporal changes at the Al-L3 edges, indicative for metallic Al [34], could 
be observed in the EELS time series of the Al-Lo3 edges in Fig. 10b. Since the TEM 
sample preparation was done without Ar ions, the origin of the incorporated Ar 
can be attributed to the sputter gas of the deposition process. We assume, Ar atoms 
implanted during reactive HfO, sputtering, where the Al and the top-most Nb back 
electrode were oxidized, as no traces of Ar could be found in the metallic Al thin 
film. 

The extensive analysis of a memristive device examined various aspects. These 
include the oxidation and roughness of the Nb bottom electrode as well as different 
oxidation states of the NbO, in dependence on the position of the device on the 
wafer which lead to malfunctions. However, the switching mechanism could not 
be resolved with TEM due to its interfacial character with very small dimensions 
and amorphous structure. Last but not least, gas inclusions inside the 7 nm Al,O3 
consisting of Ar and most probably molecular O2 were indicated by EDX and EELS, 
whether these are important for the resistive switching will be investigated in future. 


5 Invasive TEM Measurements 


The TEM utilizes the interaction of accelerated electrons with solid matter to facilitate 
imaging with sub-A spatial resolution. Due to the highly energetic impact, sensitive 
materials can be severely damaged by prolonged irradiation. Especially at high dose- 
rates, the induced beam damage can limit the resolution and available time for TEM 
analysis. The following chapter introduces a broad overview of different mechanisms 
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of beam damage, which is followed by practical examples from memristive devices, 
and reasons, why they require special care. 


5.1 Beam Damage Overview 


Electron beam damage typically describes irreversible changes in the investigated 
sample during TEM analysis. A variety of phenomena can occur, like surface sput- 
tering, phase transformations, (re)crystallization or amorphization, valence changes 
and gas bubble formation. In general, beam damage can occur via two different 
mechanisms, which may develop simultaneously and often cannot be clearly distin- 
guished. The first mechanism is the so-called knock-on damage, where collisions 
of electrons cause an elastic displacement of atoms. The second mechanism occurs 
due to the inelastic interaction of beam electrons with core electrons, leading to the 
formation of an excited state. This effect is called radiolysis. Electron beam damage 
in the TEM was reviewed by Egerton [42] and Jiang [43]. This chapter discusses 
important aspects of beam damage in relation to the materials used in memristive 
devices and how it may be prevented. 

Knock-on damage occurs preferentially at the surface due to the lower binding 
energy of surface atoms, and preferential sputtering is expected in compounds of the 
elements with the lower binding energy [42]. As a ’rule of thumb’, knock-on dam- 
age increases with increasing beam energy while the effects of radiolysis decrease. 
Radiolytic damage occurs predominantly in dielectrics with low conductivity, as the 
excited state has a longer lifetime to produce point defects compared to conduc- 
tive materials. In the vicinity of the beam the excited states accumulate and create 
a charged area - typically positively charged in TEM due to secondary and Auger 
electron emission. This positive charge can lead to cation migration away from the 
irradiated area and, e.g., the creation of molecular oxygen [23, 34]. 

Different parameters influence the beam damage. As mentioned before, the accel- 
eration voltage determines which of the two damage mechanisms dominates. With 
increasing acceleration voltage an increased maximum energy can be transferred dur- 
ing interaction with an atomic nucleus leading to displacement and thus knock-on 
damage. On the other hand, inelastic scattering is reduced which reduces charging in 
dielectrics and the number of atoms in an excited state. Dielectrics typically charge 
up positively in the TEM [44], as most fast electrons are transmitted through the 
sample and secondary (SE) and Auger electrons are created. In thicker samples or 
especially in the SEM this positive charge can be compensated by absorbed elec- 
trons from the primary beam leading to negative charging [45]. The cross-section 
for SE emission has a maximum at several keV and decreases with the acceleration 
voltage, as the mean free path of the electrons increases [46]. Accordingly, charging 
and thereby radiolysis can be reduced with higher acceleration voltages and particu- 
larly thin specimens provide a further means of mitigation. Beam damage is further 
affected by the dose (accumulated electrons per unit area) as well as the dose-rate 
(accumulated electrons per unit area and unit time or also called current density of 
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the electron beam). The excited state requires sufficient energy for atomic displace- 
ments, while the life-time and localization need to be high enough so that an atom 
can move after the energy is converted into momentum [44]. Accordingly, radiolysis 
mainly affects insulators, as lifetime (and localization) are too short in conductors, 
where the excited state is rapidly relaxing. However, in insulators excited atoms may 
change their valence state or start to move. For reversible processes (in the sense 
of damage and recovery), the dose-rate determines the damage. Significant damage 
only occurs if the activation is faster than the recovery. In this case, it is sufficient to 
reduce the dose-rate to prevent beam damage, although the accumulated dose after 
prolonged measurements can be high [32]. However, it is not necessarily the case 
that the recovered and the original state are identical. The simplest way to reduce the 
dose-rate is to analyze the sample at low magnification. In case, high resolution is 
relevant, the beam current (spot size) needs to be reduced until no damage occurs. 
The following chapter describes examples of beam damage occurring in different 
material classes used for memristive devices. 


5.2 Thin Film Metal Layers in TEM 


Metals mainly suffer from knock-on damage, which can be easily reduced by employ- 
ing low acceleration voltages. However, most metals are commonly covered with a 
thin oxide layer, and in the case of silver even with a sulfide layer. These compounds 
can be decomposed by the electron beam, leaving mobile silver nanoparticles on 
the surface. TEM experiments were performed at the tip of an intact atomic force 
microscope (AFM) apex. The apex is coated with a gold thin film as electrode, a 
silver thin film as active metal and a SiN,-matrix as insulating layer. This so-called 
Memtip-approach enables the investigation of the very same conducting filament 
with different counter-electrodes [47]. The TEM experiments revealed the invasive 
character of the electron beam making ex situ measurements before and after con- 
ductive AFM (c-AFM) not feasible. After 15 min of electron beam irradiation with 
200 kV acceleration voltage silver protrusions and morphological as well as chemi- 
cal changes in the SiN,-matrix had developed (see Fig. 11a, b). The matrix became 
denser and thinner, and in EDX, no nitrogen could be detected. Instead, oxygen 
was found, which suggests that the pristine matrix was porous and did not entirely 
protect the silver from the atmosphere. It is likely that atmospheric sulfur formed 
some Ag»S under the matrix, which is known to build Ag protrusions under electron 
beam irradiation due to mobile Agt ions inside Ag2S and injected electrons [48— 
50]. Accordingly, samples containing silver should be transferred and stored with 
particular care prior to analysis, e.g., by using a vacuum transfer holder. A different 
approach is to seal the surface in situ with a non-porous matrix to prevent oxidation 
and surface migration, as reported in [51]. Otherwise, such morphological changes 
can lead to the misinterpretation of ex situ or in situ biasing TEM experiments since 
these morphological changes can already be formed by contact with the atmosphere 
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Fig. 11 AFM apex before a and after b TEM and EDX measurements. In b, arrows on the right 
indicate evolving Ag protrusions and arrows on the left indicate morphological and chemical changes 
of the SiN, matrix [47] 


or by the electron beam. Accordingly, ex situ or in situ TEM experiments should be 
reproduced without electrical stimuli. 


5.3 Oxides in the TEM and SEM 


The key component in a multitude of vertical memristors is a dielectric thin film 
[52]. Because of this, special care should be taken when analyzing these dielectrics 
in the TEM, and, of course, also during imaging and preparation in the SEM. The 
following section describes various phenomena that can occur with oxides in the 
electron beam, especially when they are produced with high defect concentrations. 


5.3.1 Crystallization of Amorphous Oxides 


The active oxide in memristors being amorphous can be beneficial, as the amorphous 
state is isotropic in comparison to the granular microstructure of polycrystalline 
thin films. Further, the processing costs can be reduced due to the lower deposition 
temperatures of amorphous films. As a model system to be analyzed, amorphous 
hafnia was chosen, which represents one of the most studied high-k oxides in many 
fields, including resistive devices [53, 54] as well as ferroelectric applications [55]. 

Mittmann et al. observed that radio frequency (RF) sputtering of hafnia leads to 
sub-stoichiometric, amorphous hafnia thin films, which can crystallize in the ferro- 
electric orthorhombic (Pca2,) phase after rapid thermal annealing (RTA). However, 
when sufficient oxygen is added during the deposition, the monoclinic equilibrium 
phase is obtained [55]. We observed the crystallization of the pristine amorphous 
hafnia thin films during the TEM measurements, which appeared more pronounced 
at higher magnification (i.e., with higher dose and dose rate). Figure 12a shows such 
a crystallized region after 15 min of imaging in high-resolution mode. The crys- 
tallization starts from the very beginning of electron beam irradiation, though. The 
selected area diffraction (SAED) inset indicates a polycrystalline phase-mixture of 
hafnia. This thin film was RF sputtered with Ar from a hafnia target after a condi- 
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tioning time of 10 min to reach a steady state on the target surface. However, when 
the target was pre-treated with a mixed oxygen-argon-plasma to increase the oxygen 
content, the thin film exhibited few nanocrystallites in the pristine state which could 
serve as nucleation center for crystallization. Nevertheless, no crystallization caused 
by the electron beam was observed (see Fig. 12b). We hypothesize that the chemi- 
cal composition has a significant effect on the beam-induced crystallization of the 
investigated hafnia. The same is true for the crystalline phase after RTA. Admixture 
of sufficient oxygen leads to the monoclinic phase while a sub-stoichiometric hafnia 
can crystallize in the orthorhombic phase [55]. 

Investigations with different substrates were performed to evaluate their influence 
on the crystallization, e.g., the influence of charging and oxygen-absorption. These 
substrates include amorphous carbon, amorphous Si3N, and polycrystalline TiN, 
which is also serving as back-electrode in metal-insulator-metal (MIM) structures. 
On the amorphous carbon layer of a standard TEM grid a lot of nanocrystallites were 
present as-deposited, while the electron beam still induced strong crystallization of 
the thin film (see Fig. 12c). The same is true for hafnia on polycrystalline TiN thin film 
(see Fig. 12d). These findings underline that the nanocrystallites are no prerequisite 
for electron beam crystallization. Of higher importance is the stoichiometry of the 
thin film. 

Further, a simple heating effect as cause for the crystallization in the electron 
beam can be ruled out. After prolonged heating instead of electron irradiation a pure, 


Fig. 12 TEM images of an amorphous, RF-sputtered HfO, thin film on various substrates after up 
to 15 min of electron beam irradiation. In a and b, the thin film was sputtered on a Si3N4 membrane, 
in c on amorphous carbon and in d on a Siz N4/TiN thin film stack. In b, the thin film is showing no 
crystallization in the electron beam, but a few nanocrystals are present in the pristine state. In this 
case, the ceramic target was pre-sputtered with oxygen admixture. The insets show SAED patterns 
with diffraction rings of TiN in d beeing indicated with red arcs 
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thermodynamically stable monoclinic HfO% is expected [56]. In our examination, 
the thin films with higher oxygen content should crystallize as well due to a heating 
effect, which is not the case. In addition, several studies indicate that the temperature 
increases by no more than 10 °C during normal imaging conditions [42, 44, 57-59]. 

As acause for the crystallization, radiolysis seems to be likely, as different studies 
reported on the occurrence of crystallization when amorphous oxides were reduced 
in the electron beam [60, 61]. 

However, this beam crystallization is not affected by whether the substrate is 
conductive (TiN and amorphous carbon) or insulating (Si3N4). Accordingly, a sub- 
sequent charging effect does not play a major role for the crystallization. 

Taking these investigations into account, we propose that only the initial oxygen 
content, or, in terms of oxygen vacancies, the defect concentration, determines the 
crystallization of the amorphous HfO, in the electron beam and also the resulting 
phase after annealing. However, an unambiguous statement is hardly possible. Fur- 
ther triggers for crystallization could be, e.g., the amorphous structure, free volume 
and presence of nucleation centers. 

A second example for defect concentration-dependent beam damage was already 
discussed in Sect. 2. Here, Ar-milling with energies above 3 keV can alter crystalline 
Al,O3 in a way that it becomes susceptible to beam damage [14]. These findings 
highlight that the defect-concentration has a major impact on the stability of the 
sample in the TEM. This could be explained by the increasing free volume due to 
the defects leading to higher diffusion rates due to lower activation energies. On the 
other hand, beam damage produces defects. One example is a ferroelectric BaTiO} 
thin film that can be converted into a memristive thin film with configurable resistive 
states by electron beam irradiation [62]. 


5.3.2 Hole-Drilling and Oxide Reduction 


In STEM mode, the focused electron beam has a high dose-rate (current-density), 
which can be more destructive in comparison to the TEM mode. In Fig. 13a, the 
electron beam produced a line of holes after an EDX line scan across a HfO> thin 
film with a dwell time of 6 s at each position. Close to the dark area of the holes in 
the STEM image, high intensity in the vicinity gives an indication for the formation 
mechanism of these holes. Cazaux proposed a possible mechanism, starting with the 
accumulation of positive charges, which occur due to secondary and Auger electron 
emission in the vicinity of the electron beam [35]. These positive charges repel 
cations, creating an excess of anions which can react to molecular oxygen [23, 34]. 
Following this mechanism, the bright Z-contrast next to the holes in Fig. 13a may 
hint towards a thicker crater rim with Hf-excess. 

Such a reduction mechanism is dose-rate-dependent so that below a certain, 
material-dependent threshold, no visible beam damage occurs [32] and recovery 
processes can compensate the damage [33]. 
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Fig. 13 a STEM image showing hole drilling in an amorphous HfO> thin film after a line scan. b 
HRTEM micrograph and FFT-inset reveal crystalline regions around the holes 
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Fig. 14 a and b EFTEM (at 17 eV energy loss) images of a PECVD-deposited SiOz thin film after 
doses of 1.3 and 22 Me/nm”, respectively. In b, silicon-rich nanoparticles develop at the bright dots. 
c Electron beam-induced conductivity changes of the same SiOz film. Current was measured at 3 
V. Adapted with permission from Cambridge University Press [63] 


The following section describes a tool which can be used to measure this thresh- 
old in situ in the TEM or SEM. This threshold becomes particularly important when 
defect concentrations or defect variations are to be measured (e.g., using EELS). 
The (partial) reduction of oxides is equivalent to the generation of oxygen vacancies, 
which are commonly discussed point defects involved in the memristive switching 
mechanism. In addition, the thresholds for stoichiometric oxides reported in literature 
[42] will vary significantly from defective oxides. For this purpose, in situ electri- 
cal measurements on several thin film oxides were carried out, revealing reversible 
changes in conductivity attributed to a reduction in the electron beam and subsequent 
re-oxidation of the materials (see Fig. 14c). In case of SiO, thin films, the electron 
dose-induced formation of Si-rich particles leads to an increase in conductivity [63]. 
This effect can be reversed upon exposure to the atmosphere and even in the vacuum 
of the TEM when the beam is blanked. In the blue regions of Fig. 14c, the conduc- 
tivity is increased during electron beam irradiation, while in the yellow areas, the 
conductivity is decreased upon blanking the beam. The chemical changes are indi- 
cated by energy-filtered (EF)TEM dose series showing Si nanoparticle formation by 
filtering the Si plasmon peak at 17 eV. Applying a dose of 22 Me/nm? white dots have 
evolved gradually in Fig. 14b indicating the formation of particles. The conductivity 
change caused by irradiation can be utilized to determine the critical dose for beam 
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damage, thus providing data for the inhibition of such effects. This can be used prior 
to in situ or in operando TEM and SEM experiments with ceramics to prevent beam 
effects on the I(V) characteristics and morphology. Due to this reversible behavior 
of the electron beam on the I(V) characteristics we labeled this experiment as the 
TEMristor. In these measurements, the beam damage was dominated by radiolysis, as 
indicated by the dependence on the acceleration voltage. The critical dose decreases 
with the acceleration voltage. Especially in the SEM with voltages of 30 kV and 
lower, e.g., TiO, exhibits a lower critical dose compared to 300 kV in the TEM. 

This observation reconnects to the TEM preparation in Sect. 2 and highlights 
that dielectrics may have already been damaged by the electron beam during FIB- 
preparation. An effect that should be reduced by applying low beam-dose as well as 
a conductive coating on the sample. 


5.4 Conclusions of Electron Beam Effects 


In this chapter different mechanisms for beam damage in the TEM were described. 
Metallic samples are more prone to knock-on damage while dielectrics are more 
prone to radiolysis. In the context of memristive devices with high defect concen- 
trations, either specially-designed or produced on purpose through a formation step, 
beam damage is a major concern: The included defects facilitate the beam damage, 
especially in the regions of interest where the memristive switching occurs. Low- 
dose rates should therefore be chosen to investigate memristive devices, either to 
prevent or at least reduce the beam damage. In situ current measurements can be 
used to determine the threshold below which no significant damage occurs. Further 
aids and best practices will be described in the next chapter. 


6 Solutions 


Materials can be sensitive to damage induced by either TEM sample preparation via 
FIB and Ar ion milling, or even the TEM measurement itself — this holds especially 
true for thin films designed for memristive devices, i.e., in non-equilibrium states 
and with high defect concentrations. In this section, different approaches for reliable 
TEM analysis of memristive devices are proposed. 


6.1 Ex situ Measurements on Horizontal Devices 


The best way to verify the reliability of TEM measurements of memristive devices, 
and to exclude severe alterations by the preparation or the electron beam is to elec- 
trically characterize the TEM specimen before and after in an ex situ manner. This 
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Fig. 15 Overview STEM image before a and after b ex situ electrical probing. Bimetallic AgPt 
nanoparticles were deposited by a Haberland cluster source on top of the electrodes and these were 
covered with a SiON-matrix. The inset in a shows the I(V)-characteristics of the dielectric break 
down. The schematic design of the ex situ grids with structured Au contracts on Siz3N4 membranes is 
illustrated in c. An EDX linescan in d along the orange arrow reveals that the fractal dendrites consist 
mainly of Ag. The position of d is indicated by the orange square in b. Adapted by permission 
from [Springer Nature Customer Service Centre GmbH]: [Springer] [64] 


approach can be realized via a horizontal electrode design which can be lithographi- 
cally structured on electron-transparent Si;N4, membranes. Such an electrode design 
is illustrated in Fig. 15c. The detailed procedure is described in reference [64]. By 
means of electron beam lithography electrode distances of 40 nm and below were 
achieved, which enable a good observation of the region of interest at high resolu- 
tion. The corresponding prototype, deposited via, e.g., sputtering, can subsequently 
be analyzed in the TEM. Following this approach, it was possible to observe fractal 
growth in an AgPt nanoparticle film below the percolation limit, encapsulated in a 
SiON-matrix which suffered a dielectric breakdown during electrical probing. EDX 
measurements in Fig. 15c revealed that these fractal dendrites consist of silver and 
leave behind Pt-rich nanoparticles after electrical biasing. This observation provides 
information for the switching mechanism of a similar, but vertical nanoparticle-based 
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diffusive memristor [65]. In bimetallic nanoparticles, the less noble metal mainly 
contributes to the filament formation while the noble counter part remains behind 
and can serve as an anchor point for the filament to enhance memristive switching 
[21]. However, this ex situ approach may be insufficient, as a (partially) reduced 
oxide might re-oxidize during transfer to the ex situ measurement setup and electron 
beam damage could go unnoticed. To be sure, in situ or in operando experiments are 
necessary, and suitable approaches are described in the following section. 


6.2 FIB Preparation for In situ Experiments 


A horizontally-arranged prototype device may serve well for TEM measurements 
and fundamental research, but is not comparable to real devices, as the geometry 
of the electrodes has a major impact on the properties of the device [66]. For this 
purpose, the real devices have to be analyzed and to exclude preparation artefacts, 
the functionality of the TEM sample should be checked and compared to the device. 
TEM sample preparation for in operando experiments on vertical devices requiring 
FIB turned out to be a major obstacle due to Pt and Ga contamination and material 
re-deposition, all creating possible short circuits. To overcome these problems, the 
sample geometry and a specially adapted MEMS chip for in situ electrical biasing 
were designed by Zintler et al. [67]. The preparation improvements start with a spe- 
cial stub, mounting both the sample and the MEMS chip orthogonal to each other. 
This stub enables venting-free preparation including lift-out, transfer, contacting of 
the electrodes and thinning. Top and bottom electrodes are contacted with Pt trenches 
from the gas injection system in the FIB instrument. To prevent short circuits and 
memristive switching apart from the electron transparent region, the counter elec- 
trodes are cut next to the Pt trenches by means of Ga-ion etching. The whole prepa- 
ration is described in detail in [67]. After successful preparation, the lamella can be 
electrically characterized in situ or even in operando in the TEM and the comparison 
of the lamella device with the macro device shows striking similarities (compare 
Fig. 16b, c with f, e). Further, the influence of the electron beam can be measured 
electrically. 

Despite these striking similarities, the approach also presents some drawbacks. 
First, the preparation is time-consuming and is likely to end up in a short circuit. 
Secondly, the lamella device is quite large, and the resistive switching therefore 
difficult to observe, as it may occur at any point in the lamella. This second point is 
not only a drawback, as the geometry is not altered significantly and a larger area 
can be observed. Further, such a lamella has two free surfaces which may dominate 
the switching when diffusion or migration is involved. And last but not least, the 
small dimensions make Joule heating more critical which may lead to a faster device 
degradation and could limit TEM observations [67]. 
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Fig. 16 a FIB lamella mounted and contacted by Pt trenches to contacts of the MEMS chip. d 
STEM image of the device for the in situ measurements. The inset shows a Bright-field STEM 
image with higher magnification. b and ¢ show I(V)-characteristics of the lamella device while f 
and e represent data for the macroscopic device. b and f show the gradual reset and c and e the 
unipolar resistive switching. Adapted with permission from Elsevier [67] 


6.3 Conclusions and Best Practice 


In conclusion, reliable TEM experiments on memristive devices can be realized by 
adhering to the following guidelines: 


Surface diffusion processes must be prevented, e.g., by the application of barrier 
layers on the top and bottom of the devices, cf. [51]. 

Contamination of the samples must be prevented, e.g., by using suitable device 
transfer systems from the sputtering devices to the TEM. 

If possible, check for functionality before and after preparation and after the TEM- 
measurements. For this purpose, horizontal instead of vertical devices should be 
produced and analyzed. These systems do not require sophisticated TEM prepa- 
ration, since they can be fabricated directly for TEM analysis. 

For real device characterization, electrically contacted FIB lamellas should be 
analyzed in operando and the I(V)-characteristics should be critically compared 
to the real device. 

In ex situ, in situ or in operando, experiments, current measurements should be 
used to determine the threshold for electron beam damage. If this is not possible, 
the usage of low dose rates but high acceleration voltages is advisable, if dielectric 
memristive devices are to be characterized. 
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7 Experimental 


TEM measurements were performed on different microscopes. High-end EELS for 
the DBMD characterization was done on a FEI Titan? provided by ESTEEM3 facil- 
ities. Acceleration voltages of 300 kV and 80 kV were used in conjunction with a 
monochromator, XFEG, GIF quantum energy analyzer and SuperX EDX detector. 
The analysis of the electron beam crystallization of hafnia thin films was conducted 
on a JEOL JEM-2100 with 200 kV and a LaBg cathode. Hole-drilling experiments, 
also with hafnia thin films were performed using a Tecnai F30 G* microscope with 
300 kV and a field emission gun. 


Acknowledgements This project has received funding from the European Union’s Horizon 2020 
research and innovation programme under grant agreement No 823717 — ESTEEM3. The financial 
support by the German Research Foundation (Deutsche Forschungsgemeinschaft—DFG) through 
FOR 2093 is gratefully acknowledged. Funded by the Deutsche Forschungsgemeinschaft (DFG, 
German Research Foundation) — ProjectID 434434223 — SFB 1461. 


References 


1. Kendall, J.D., Kumar, S.: The building blocks of a brain-inspired computer. Appl. Phys. Rev. 
7, 011305 (2020). https://doi.org/10.1063/1.5129306 

2. Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R., Eleftheriou, E.: Memory devices and 
applications for in-memory computing. Nat. Nanotechnol. 15, 529-544 (2020). https://doi. 
org/10.1038/s41565-020-0655-z 

3. Strobel, J., Neelisetty, K.K., Chakravadhanula, V.S.K., Kienle, L.: Transmission electron 
microscopy on memristive devices: an overview. Appl. Microsc. 46, 206-216 (2016). https:// 
doi.org/10.9729/AM.2016.46.4.206 

4. Yang, Y., et al.: Probing electrochemistry at the nanoscale: in situ TEM and STM character- 
izations of conducting filaments in memristive devices. J. Electroceramics 39, 73—93 (2017). 
https://doi.org/10.1007/s10832-017-0069-y 

5. Sun, W., et al.: Understanding memristive switching via in situ characterization and device 
modeling. Nat. Commun. 10 (2019). https://doi.org/10.1038/s41467-019-11411-6 

6. Xu, H., et al.: Dynamic structure-properties characterization and manipulation in advanced 
nanodevices. Mater. Today Nano 7, 100042 (2019). https://doi.org/10.1016/j.mtnano.2019. 
100042 

7. Yang, X., et al.: A review of in situ transmission electron microscopy study on the switching 
mechanism and packaging reliability in non-volatile memory. J. Semicond. 42, 013102 (2021). 
https://doi.org/10.1088/1674-4926/42/1/013102 

8. Ielmini, D., Waser, R. (eds.): Resistive Switching: From Fundamentals of Nanoionic 
Redox Processes to Memristive Device Applications. Wiley (2015). https://doi.org/10.1002/ 
9783527680870 

9. Du, H., et al.: Nanosized conducting filaments formed by atomic-scale defects in redox-based 
resistive switching memories. Chem. Mater. 29, 3164-3173 (2017). https://doi.org/10.1021/ 
acs.chemmater.7b00220 

10. Kwon, D.-H., et al.: Atomic structure of conducting nanofilaments in TiO2 resistive switching 
memory. Nat. Nanotechnol. 5, 148—153 (2010). https://doi.org/10.1038/nnano.2009.456 
11. Li, C., et al.: Dynamic observation of oxygen vacancies in hafnia layer by in situ transmission 

electron microscopy. Nano Res. 8, 3571-3579 (2015). https://doi.org/10.1007/s12274-015- 
0857-0 


154 


12. 


13. 


16. 


17. 


18. 


19. 


20. 


21: 


22. 


23. 


24. 


25. 


26. 


27. 


28. 


29. 


30. 


31. 


32. 


O. Gronenberg et al. 


Jang, J.H., et al.: Investigation of oxygen-related defects and the electrical properties of atomic 
layer deposited HfO> films using electron energy-loss spectroscopy. J. Appl. Phys. 109, 023718 
(2011). https://doi.org/10.1063/1.3544039 

Zahari, F., et al.: Correlation between sputter deposition parameters and I-V characteristics in 
double-barrier memristive devices. J. Vac. Sci. Technol. B 37, 061203 (2019). https://doi.org/ 
10.1116/1.5119984 


. Nufer, S., et al.: Quantitative atomic-scale analysis of interface structures: transmission electron 


microscopy and local density functional theory. Phys. Rev. Lett. 86, 5066-5069 (2001). https:// 
doi.org/10.1103/PhysRevLett.86.5066 


. Nufer, S.: Spatially Resolved Analytical Electron Microscopy at Grain Boundaries of a-A1203 


(2001) 

Kang, H.-J., Kim, J., Oh, J., Back, T., Kim, H.: Ultra-thin TEM sample preparation with 
advanced backside FIB milling method. Microsc. Microanal. 16, 170-171 (2010). https://doi. 
org/10.1017/S1431927610054474 

Gurvitch, M., Washington, M.A., Huggins, H.A.: High quality refractory Josephson tunnel 
junctions utilizing thin aluminum layers. Appl. Phys. Lett. 42, 472-474 (1983). https://doi. 
org/10.1063/1.93974 

Ishitani, T., Yaguchi, T.: Cross-sectional sample preparation by focused ion beam: a review of 
ion-sample interaction. Microsc. Res. Tech. 35(4), 320-333 (1996). https://doi.org/10.1002/ 
(SICI)1097-0029(19961101)35:4<320::AID-JEMT3>3.0.CO;2-Q 

Strobel, J., et al.: In depth nano spectroscopic analysis on homogeneously switching dou- 
ble barrier memristive devices. J. Appl. Phys. 121, 245307 (2017). https://doi.org/10.1063/1. 
4990145 

Kiss, K.: Problem Solving with Microbeam Analysis. OCLC: 1058661531. Elsevier Science, 
Amsterdam (2014) 

Vahl, A., et al.: Single target sputter deposition of alloy nanoparticles with adjustable composi- 
tion via a gas aggregation cluster source. Nanotechnology 28, 175703 (2017). https://doi.org/ 
10.1088/1361-6528/aa66ef 

Hansen, M. et al.: A double barrier memristive device. Sci. Rep. 5 (2015). https://doi.org/10. 
1038/srep13753 

Jiang, N., Spence, J.C.: Interpretation of Oxygen K pre-edge peak in complex oxides. Ultra- 
microscopy 106, 215-219 (2006). https://doi.org/10.1016/j.ultramic.2005.07.004 
Klimenkov, M., Moslang, A., Lindau, R.: EELS analysis of complex precipitates in PM 2000 
steel. Eur. Phys. J. Appl. Phys. 42, 293-303 (2008). https://doi.org/10.105 1/epjap:2008084 
Nigo, S., et al.: Conduction band caused by oxygen vacancies in aluminum oxide for resistance 
random access memory. J. Appl. Phys. 112, 033711 (2012). https://doi.org/10.1063/1.4745048 
Bruley, J., et al.: Investigations of the chemistry and bonding at niobiumsapphire interfaces. J. 
Mater. Res. 9, 2574-2583 (1994). https://doi.org/10.1557/JMR.1994.2574 

Imamura, T., Hasuo, S.: Cross-sectional transmission electron microscopy observation of 
Nb/AIO,—AI/Nb Josephson junctions. Appl. Phys. Lett. 58, 645-647 (1991). https://doi. 
org/10.1063/1.104556 

Winkler, B., et al.: Oxygen K-edge electron energy loss spectra of hydrous and anhydrous 
compounds. J. Phys. Condens. Matter 25, 485401 (2013). https://doi.org/10.1088/0953-8984/ 
25/48/485401 

Wirth, R., Wunder, B.: Characterization of OH-containing phases by TEM using electron 
energy-loss spectroscopy (EELS): Clinohumite-OH, chondrodite-OH, phase A, and the (F, 
OH)-solid solution series of topaz. J. Trace Microprobe Tech. 18(1), 35-50 (2000) 

Garvie, L.A.: Can electron energy-loss spectroscopy (EELS) be used to quantify hydrogen 
in minerals from the O K edge? Am. Mineral. 95, 92-97 (2010). https://doi.org/10.2138/am. 
2010.3290 

Kosugi, N., Shigemasa, E., Yagishita, A.: High-resolution and symmetry-resolved oxygen K- 
edge spectra of 0 2. Chem. Phys. Lett. 190, 8 (1992) 

Jiang, N., Spence, J.C.H.: On the dose-rate threshold of beam damage in TEM. Ultramicroscopy 
113, 77-82 (2012). https://doi.org/10.1016/j.ultramic.2011.11.016 


Critical Discussion of Ex situ and In situ TEM Measurements on Memristive Devices 155 


33: 


34. 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 


43. 


44. 


45. 


46. 


47. 


48. 


49. 


50. 


51. 


52. 


Mkhoyan, K.A., Silcox, J., Ellison, A., Ast, D., Dieckmann, R.: Full recovery of electron 
damage in glass at ambient temperatures. Phys. Rev. Lett. 96 (2006). https://doi.org/10.1103/ 
PhysRevLett.96.205506 

Berger, S.D., Salisbury, I.G., Milne, R.H., Imeson, D., Humphreys, C.J.: Electron energy- 
loss spectroscopy studies of nanometre-scale structures in alumina produced by intense 
electron-beam irradiation. Philos. Mag. B 55, 341-358 (1987). https://doi.org/10.1080/ 
13642818708208619 

Cazaux, J.: Correlations between ionization radiation damage and charging effects in trans- 
mission electron microscopy. Ultramicroscopy 60, 411—425 (1995). https://doi.org/10.1016/ 
0304-399 1(95)00077- 1 

Ayoola, H.O., et al.: Origin and suppression of beam damage-induced oxygen-K edge artifact 
from gamma-A1203 using Cryo-EELS. Ultramicroscopy 219, 113127 (2020). https://doi.org/ 
10.1016/j.ultramic.2020.113127 

Arhammar, C., et al.: Unveiling the complex electronic structure of amorphous metal oxides. 
Proc. Natl. Acad. Sci. 108, 6355-6360 (2011). https://doi.org/10.1073/pnas. 1019698108 
Seeman, V., et al.: Atomic, electronic and magnetic structure of an oxygen interstitial in neutron- 
irradiated A1203 single crystals. Sci. Rep. 10, 15852 (2020). https://doi.org/10.1038/s41598- 
020-72958-9 

Fritz, S., et al.: Correlating the nanostructure of Al-oxide with deposition conditions and dielec- 
tric contributions of two-level systems in perspective of superconducting quantum circuits. Sci. 
Rep. 8, 7956 (2018). https://doi.org/10.1038/s41598-018-26066-4 

Malek, A., Eikerling, M.H.: Chemisorbed Oxygen at Pt(111): a DFT study of structural and 
electronic surface properties. Electrocatalysis 9, xs370-379 (2018). https://doi.org/10.1007/ 
$12678-017-0436-0 

Tan, E., Mather, P.G., Perrella, A.C., Read, J.C., Buhrman, R.A.: Oxygen stoichiometry and 
instability in aluminum oxide tunnel barrier layers. Phys. Rev. B 71 (2005). https://doi.org/10. 
1103/PhysRevB.71.161401 

Egerton, R.: Radiation damage to organic and inorganic specimens in the TEM. Micron 119, 
72-87 (2019). https://doi.org/10.1016/j.micron.2019.01.005 

Jiang, N.: Electron beam damage in oxides: a review. Rep. Prog. Phys. 79, 016501 (2016). 
https://doi.org/10.1088/0034-4885/79/1/016501 

Hobbs, L.W.: Murphy’s law and the uncertainty of electron probes. Micro-anal. Microlithogr. 
4, 14 (1990) 

Zhang, H.-B., Feng, R.-J., Ura, K.: Utilizing the charging effect in scanning electron 
microscopy. Sci. Prog. 87, 249-268 (2004). https://doi.org/10.3184/003685004783238490 
Williams, D.B., Carter, C.B.: Transmission Electron Microscopy: A Textbook for Materials 
Science, 2nd edn., 4 pp. Springer, New York (2008) 

Carstens, N., et al.: Enhancing reliability of studies on single filament Memristive switching 
via an unconventional cAFM approach. Nanomaterials 11, 265 (2021). https://doi.org/10.3390/ 
nano11020265 

Liang, C. et al.: lonic-electronic conductor nanostructures: template-confined growth and non- 
linear electrical transport. Small 1, 971-975 (2005). https://doi.org/10.1002/smll.200500155 
Liang, C., Terabe, K., Hasegawa, T., Aono, M.: Resistance switching of an individual Ag 2 
S/Ag nanowire heterostructure. Nanotechnology 18, 485202 (2007). https://doi.org/10.1088/ 
0957-4484/18/48/485202 

Liao, Z.-M., et al.: Resistive switching and metallic-filament formation in Ag2S nanowire 
transistors. Small 5, 2377—2381 (2009). https://doi.org/10.1002/smll.200900642 

Alissawi, N., et al.: Tuning of the ion release properties of silver nanoparticles buried under 
a hydrophobic polymer barrier. J. Nanoparticle Res. 14, 928 (2012). https://doi.org/10.1007/ 
$11051-012-0928-z 

Li, Y., Wang, Z., Midya, R., Xia, Q., Yang, J.J.: Review of memristor devices in neuromorphic 
computing: materials sciences and device challenges. J. Phys. D: Appl. Phys. 51, 503002 
(2018). https://doi.org/10.1088/1361-6463/aade3f 


156 


53. 


54. 


55. 


56. 


SYA 


58. 


59. 


60. 


6l. 


62. 


63. 


64. 


65. 


66. 


67. 


O. Gronenberg et al. 


Gritsenko, V.A., Perevalov, T.V., Islamov, D.R.: Electronic properties of hafnium oxide: a 
contribution from defects and traps. Phys. Rep. 613, 1-20 (2016). https://doi.org/10.1016/j. 
physrep.2015.11.002 

Grossi, A., Zambelli, C., Olivo, P., Crespo-Yepes, A., Martin-Martinez, J., Rodriguez, R., 
Nafria, M., Perez, E., Wenger, C.: Electrical characterization and modeling of 1T-1R RRAM 
arrays with amorphous and poly-crystalline HfO2. Solid-State Electron. 7, 187—193 (2016). 
https://doi.org/10.1016/j.sse.2016.10.025 

Mittmann, T. et al.: Origin of ferroelectric phase in undoped HfOz films deposited by sputtering. 
Adv. Mater. Interfaces, 1900042 (2019). https://doi.org/10.1002/admi.201900042 
Narasimhan, V.K., et al.: In situ characterization of ferroelectric HfO2 during rapid thermal 
annealing. Physica Status Solidi (RRL) — Rapid Res. Lett. 15, 2000598 (2021). https://doi.org/ 
10.1002/pssr.202000598 

Zhang, Y., et al.: lon-induced damage accumulation and electron-beam-enhanced recrystal- 
lization in SrTiO3. Phys. Rev. B 72, 094112 (2005). https://doi.org/10.1103/PhysRevB.72. 
094112 

Jencic, I., Bench, M.W., Robertson, I.M., Kirk, M.A.: Electron-beam-induced crystallization 
of isolated amorphous regions in Si, Ge, GaP, and GaAs. J. Appl. Phys. 78, 974-982 (1995). 
https://doi.org/10.1063/1.360764 

Lulli, G., Merli, P.G., Antisari, M.V.: Solid-phase epitaxy of amorphous silicon induced by 
electron irradiation at room temperature. Phys. Rev. B 36, 8038-8042 (1987). https://doi.org/ 
10.1103/PhysRevB.36.8038 

Kern, P., Miiller, Y., Patscheider, J., Michler, J.: Electron-beam-induced topographical, chemi- 
cal, and structural patterning of amorphous titanium oxide films. J. Phys. Chem. B 110, 23660- 
23668 (2006). https://doi.org/10.1021/jp0642589 

Kern, P., Widmer, R., Gasser, P., Michler, J.: Local tuning of conductivity in amorphous titanium 
oxide films by selective electron beam irradiation. J. Phys. Chem. C 111, 13972-13980 (2007). 
https://doi.org/10.1021/jp0711803 

Molinari, A., etal.: Configurable resistive response in BaTiO 3 ferroelectric memristors via elec- 
tron beam radiation. Adv. Mater. 32, 1907541 (2020). https://doi.org/10.1002/adma.201907541 
Neelisetty, K.K., et al.: Electron beam effects on oxide thin films-structure and electri- 
cal property correlations. Microsc. Microanal. 25, 592—600 (2019). https://doi.org/10.1017/ 
$1431927619000175 

Gronenberg, O., Carstens, N., Vahl, A., Faupel, F., Kienle, L.: In: Tiginyanu, I., Sontea, V., 
Railean, S. (eds.) 4th International Conference on Nanotechnologies and Biomedical Engineer- 
ing. IFMBE Proceedings, pp. 71-74. Springer International Publishing, Cham (2020). https:// 
doi.org/10.1007/978-3-030-31866-6_15 

Vahl, A., Carstens, N., Strunskus, T., Faupel, F., Hassanien, A.: Diffusive memristive switching 
on the nanoscale, from individual nanoparticles towards scalable nanocomposite devices. Sci. 
Rep. 9 (2019). https://doi.org/10.1038/s41598-019-53720-2 

Xia, Q., et al.: Impact of geometry on the performance of memristive nanodevices. Nanotech- 
nology 22, 254026 (2011). https://doi.org/10.1088/0957-4484/22/25/254026 

Zintler, A., et al.: FIB based fabrication of an operative Pt/HfO2/TiN device for resistive switch- 
ing inside a transmission electron microscope. Ultramicroscopy 181, 144-149 (2017). https:// 
doi.org/10.1016/j.ultramic.2017.04.008 


Critical Discussion of Ex situ and In situ TEM Measurements on Memristive Devices 157 


Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 
International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, 
adaptation, distribution and reproduction in any medium or format, as long as you give appropriate 
credit to the original author(s) and the source, provide a link to the Creative Commons license and 
indicate if changes were made. 

The images or other third party material in this chapter are included in the chapter’s Creative 
Commons license, unless indicated otherwise in a credit line to the material. If material is not 
included in the chapter’s Creative Commons license and your intended use is not permitted by 
statutory regulation or exceeds the permitted use, you will need to obtain permission directly from 
the copyright holder. 


Modeling and Simulation of Silver-Based | @ 
Filamentary Memristive Devices siete 


Sven Dirkmann, Jan Trieschmann, and Thomas Mussenbrock 


Abstract Memristive devices based on filamentary switching are investigated by 3D 
kinetic Monte Carlo simulations. The electrochemical metallization device under 
consideration consists of a stack of Ag/TiO,/Pt thin layers. By modeling the Ag 
ion transport within the solid state electrolyte, driven by the electric field and 
thermal diffusion, the dynamics of resistive switching and conducting filament 
growth/dissolution are studied. The model allows to resolve the macroscopic time 
scale of consecutive growth and dissolution cycles. It provides realistic current- 
voltage relations as observed experimentally. Simultaneously, it grants a detailed 
characterization of the influence of the electric field and the thermal heat on the 
local resistive switching dynamics. It finally provides insight into the microscopic 
physical mechanisms involved in the set and reset kinetics during switching. It is 
concluded that the force due to electromigration on Ag in the closed filament may 
not be negligible during reset process of the device. 


Keywords Memristive devices - Electrochemical metallization cells - 
Filamentary resistive switching - Kinetic Monte Carlo + Modeling and simulation 


1 Introduction 


Today’s predominant memory technology is based on the storage of electrical charge. 
In these devices relatively high energy barriers are required to prevent the loss of its 
memory state through electron tunneling currents. Due to approximately 1000 times 
larger mass of atoms compared to electrons, the tunneling of atoms to neighboring 
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states is negligible. In contrast, memristive devices are based on the principle of 
resistive switching, whereby information is stored in the device resistance. These 
memory devices are based on the change of the atomic arrangement. Therefore, 
they do not require high energy barriers and could replace currently used memory 
concepts, especially with respect to scalability and energy efficiency. 

The first memristive devices were introduced as early as the 1960s [1-3]. In 1976, 
specifically the emergence of Ag filaments as a possibility for resistive switching was 
recognized [4]. In the 1970s, Leon O. Chua further presented a theory of memristive 
systems [5, 6]. However, especially due to the rapid progress of silicon-based, inte- 
grated circuit technology, interest in memristive devices decreased from the end of 
the 1970s until new computer architectures were considered in the 1990s. In 2008, 
Strukov et al. demonstrated resistive switching in the context of the theory of mem- 
ristive systems presented by Chua [7]. Due to their versatile usability, for example as 
artificial synapses in neural networks for new types beyond von Neumann computer 
architectures or as future, non-volatile storage elements, research on memristive 
devices is steadily increasing to this day [8—10]. 

Memristive devices are often used in pulsed mode operation. However, direct cur- 
rent operation and its related current-voltage (IV) characteristics provide important 
insights into the physical behavior of individual devices. For instance, on the basis 
of the IV characteristics a distinction can generally be made between unipolar and 
bipolar devices. In the case of bipolar switching devices, the voltage polarity in the 
set and reset process differs, whereas for unipolar devices it is the same. In addition to 
this very general classification, memristive devices can also be differentiated based 
on their physical behavior. For example there are memristive devices based on mag- 
netic effects (such as magnetic tunnel devices [11]), on electrostatic effects (such as 
the trapping effect of electrons [12]). The memristive devices particularly considered 
in this work are based on a change in the atomic configuration. Even within this type 
of memristive systems, the exact physical mechanisms of resistive switching can be 
very different and range from diffusion and migration of ions [13] through chemical 
reactions to thermal effects [14] caused by Joule heating [15]. The exact switching 
behavior is often not understood and thus prevents a reliable integration of these 
devices in electronic circuits. As a theoretical approach, this chapter is henceforth 
dedicated to the modeling and simulation of electrochemical metallization (ECM) 
cells that exhibit filamentary switching behavior. 


2 Electrochemical Metallization Cells 


Typically, ECM cells consist of a solid electrolyte with poor electrical conductivity, 
embedded between a chemically active and an inert electrode. The active electrode 
usually consists of Ag, Cu, or Ni. In addition, resistive switching has already been 
demonstrated with active electrodes made of alloys containing Ag, Au, Cu, or Ni, 
like Au/Ag alloys [16]. In addition to the chemical inactivity, another requirement 
for the inert electrode is the poor miscibility with atoms of the active electrode. These 
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Fig. 1 Scheme of the ECM cell with incipient filament growth and illustration of the processes 
implemented in the simulation: (I) oxidation, (II) surface diffusion on the Pt electrode, (III) diffusion 
in the TiO, , (IV) reduction on Ag, (V) reduction on a step, (VI) nucleation. Reprinted from [14], 
with the permission of AIP Publishing 


requirements are met by materials like Pt, Ir, and W. A wide variety of materials have 
been demonstrated as possible electrolyte materials, such as TiO2, SiO2, Ge,S, [17]. 

In this work, the ECM devices based on the Ag/TiO,./Pt materials system is inves- 
tigated with the help of simulations [18]. Its general functionality is schematically 
presented in Fig. 1. Assume that the ECM cell is in the high-resistance state (HRS) 
initially. In the set process, the chemically active Ag electrode is subject to a positive 
voltage (with respect to the opposing grounded electrode). This leads to oxidation 
of the Ag electrode following the reaction, 


Ag > Agt +e7 (1) 


with monovalent Agt ions [see Fig. 1 (D)] resulting in a partial dissolution of the 
active Ag electrode. Along with a related transfer of charge through the electrode, 
the Ag” ions are subject to the electric field in the solid electrolyte. 

Due to this electric field, the Agt ions move through the TiO, toward the Pt 
electrode [see Fig. 1 (IID]. When the Ag+ ions reach the Pt/TiO, interface, they 
can undergo different processes. Since Ag and Pt are almost immiscible at room 
temperature (see phase diagram Ag-Pt), surface diffusion of the Agt ions occurs at 
the Pt/TiO, interface [see Fig. 1 (ID]. In addition, Ag* ions may form stable nuclei 
at the Pt electrode after reduction [19]. At these nuclei, additional Ag™ ions may be 
reduced based on the following reaction [see Fig. 1 (IV)-(V)]: 


Agt +e —> Ag (2) 


This reaction leads to the growth of Ag filaments from the Pt electrode through the 
TiO, electrolyte towards the Ag electrode. The two-step process of (i) formation of 
small Ag clusters and (ii) the reduction of Ag* ions on these clusters by electron 
transfer has been proven experimentally in 1999 [20]. 

If the distance between the filament and the Ag electrode decreases, the Ohmic 
resistance also decreases (increases the probability of electron tunneling). This leads 
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to an increase in the current through the device, which is often intentionally limited 
to a maximum value. This value of maximum current signifies whether there is a 
gap between the filament and the electrode, or the filament has formed up to the Ag 
electrode and begun to develop a galvanic contact. In this state, the device is in the 
low-resistance state (LRS). To reset the device to the HRS, the voltage polarity must 
be reversed. Depending on the current through the filament, it partially dissolves due 
to the electric field, supported by an increased temperature by Joule heating and by 
electromigration. The distance between the source and the sink of ions is determined 
by the thickness of the solid electrolyte and may therefore be very small. That may 
lead to desired properties like fast switching times and a very good scaling potential 
(<10 nm). In addition, only small voltages are required for device operation due 
to the small electrolyte thickness and the correspondingly very large electric field 
strengths. A very small energy consumption (< pJ) results. Despite these advantages 
and the technical feasibility of these devices, the randomness of the inherent physical 
processes remains an issue. 

To gain a reliable understanding of the resistive switching of ECM cells, simu- 
lations on the atomic scale are indispensable. The multitude of different simulation 
models proposed so far range from concentrated models to molecular dynamics mod- 
els. This also includes continuous models and kinetic Monte Carlo (KMC) models 
in 1D and 2D [21-26]. These models aim to describe resistive switching on various 
time scales. Since filament growth is a three-dimensional phenomenon, however, a 
3D kMC simulation model is proposed as part of this work. The following central 
questions should be addressed: (i) How does filament growth proceed in ECM cells, 
(ii) what are the central influences that lead to the reset of ECM cells, and (iii) how 
does the distribution of atoms affect the resistance of the ECM cells? 


3 Simulation Scenario and Simulation Methods 


To simulate the characteristics of a real ECM cell, only a representative section 
of the device is considered in the model. The simulation box consists of a square 
base with an area of 40 nm x 40 nm, a 10 nm thick TiO, solid electrolyte and a 
3 nm thick section of an active Ag electrode. The Ag electrode consists of 38,400 
individual atoms, whereas the opposite inert Pt electrode was modeled as a boundary 
for Ag* ion motion to compensate for the poor miscibility of Ag and Pt. At all other 
interfaces (i.e., the sides), periodic periodic boundary conditions were used for ion 
motion. Occupiable, stable ion positions were represented by a cubic primitive lattice 
with a lattice constant of 5 A. The periodic lattice is reasoned by the corresponding 
short range order, which exists also in amorphous materials. 

In titanium oxide, silver is preferably present as a monovalent ion at interstitial 
sites. It can diffuse through the bulk TiO, under the influence of electric fields 
[27, 28]. The processes presented in Fig. 1 were taken into account also in the 
simulation, as these are identified as the primarily important processes. The rate of 
diffusion of Agt ions through TiO, is given by 
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Table 1 Parameters for the simulation of ECM cells 
Symbol | Quantity Value References 
OAg Conductivity of Ag 6.3 x 107 Sm7! 
OTIO, Conductivity of TiO, 1.0 x 10? Sm7! 
Enor Activation energy for oxidation 0.65 eV 
Eared Activation energy for reduction (surface/step) 0.62/0.58 eV 
Fanuc Activation energy for nucleation 0.81 eV [29] 
Ea ditt Activation energy for diffusion of Ag+ 0.61 eV 
Easurf Activation energy for surface diffusion of Agt 0.59 eV 
PAg Mass density of Ag 10,490 kgm~? [30] 
PTO, Mass density of TiO2 4,230 kgm~3 [31] 
CAg Heat capacity of Ag 235 Jkg-!k~! [32] 
CTiO, Heat capacity of TiO2 700 Jkg-!K7! [33] 
dag Thermal conductivity of Ag 429 Wm7!K7!__ | [34] 
ATIO, Thermal conductivity of TiO2 7Wm7!K7! [35] 
Eat + 0.5z;e(b; — i) 
Kitt = Vo exp ( kT (rt) | (3) 


Here, vo is the phonon frequency, E, aitt is the activation energy for diffusion of Agt 
ions through TiO,, ¢; and @; are the potential at positions indicated by i and j, z; is 
the charge number, e is the elementary charge, T is the local temperature, and ky is 
the Boltzmann constant. Since diffusion at the material interfaces usually proceeds 
at higher velocities than in the volume, a smaller activation energy Ea sure is used for 
surface diffusion compared to volume diffusion. All relevant simulation parameters 
are summarized in Table 1. 

The rates for reduction and oxidation can be derived from the Butler-Volmer 
equation [24]. One finds 


Ea red — &ozie Ag 
kred = So nla 4 
d = Vo exp ( RTE, D ) (4) 
Ea,ox = el = a )zie Ad 
Kox = -— 5 
Vo exp ( ITO.) (5) 


with the charge transfer coefficient aj = 0.5 and the activation energies Earea and 
Ex for reduction and oxidation. 

The potential difference A¢ at the electrode/TiO, and filament/TiO, interface is 
given as Ag = Ptio, — Pelectrode and Ad = orio, — Pfilament, respectively. The pro- 
cess of heterogeneous nucleation depends on its environment. Thus, the time constant 
for nucleation can be given as a function of the Ag concentration. Here, the nucle- 
ation of Ag* ions is modeled analogously to [26] as a one-particle process where 
the nucleation process is independent of the material concentration. This ensures a 
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reduction in simulation time for the very time-consuming 3D simulations. The fact 
that a critical Ag concentration must first be reached so that nucleation becomes 
probable and the resulting slowing down of the process is represented here by an 
increased activation energy. In these simulations the following nucleation rate was 
used: 


(6) 


Fanuc — agzieAd 
Knuc = Vo exp 


kT (r, t) 


with the activation energy Eanuc for nucleation. The correctness of this modeling 
approach may be confirmed experimentally afterwards from the activation energy 
for nucleation. The time it takes to set a filament (tset) decreases exponentially for 
high voltages with the applied voltage. This is reflected in the equations of the process 
rates in Arrhenius form. tse becomes infinite for small voltages [36]. Therefore, for 
every oxidation step additionally the voltage-dependent condition 


GR log [cos(é Vappi) | (7) 


must be satisfied, with the uniformly distributed random number ¢ € [0, 1], the fitting 
parameter &, and the applied voltage Vappi- 

As an aside, it is well known that in TiO,, vacancies move in the electrical field 
and can form conductive filaments. Depending on the kinetics of silver ion migration 
and vacancy migration, the latter may have an effect on the electric field distribution 
and hence on the formation of silver filaments. However, this mechanism is neglected 
in this work. 

All mentioned processes depend on the electric field. It can be calculated based on 
the continuity equation V - j = 0. Here, the displacement current has been neglected. 
With the help of a simple scale analysis, it can be shown that this assumption is 
justified. It turns out that the displacement current scales with we L? /c?T?. Therein, 
L and T are the typical length and time scale of the system, respectively. With L « 
1 nm and T « 1 s one finds that we L*/c?T? < 1. 

Electron transport through the device is dominated by Ohmic conduction. In 
the literature, the tunneling current between the filament and the electrode is often 
reported as the dominant transport mechanism [19]. To show that electron transport 
is not dominated by tunneling electrons here, the Ohmic current is compared to 
the tunneling current at the smallest possible distance resolved in the simulation of 
0.5 nm. This distance is chosen because the tunneling current scales exponentially 
with the thickness of the tunnel barrier, but the Ohmic current only linearly. Thus, 
for the minimum possible distance, the largest ratio between tunneling current and 
Ohmic current Jtun/Johm is expected. The tunneling current density may be calculated 
using the Simmons equation [37]. The height of the tunnel barrier was chosen to be 
Ppp = Pro, /ag = 0.36eV [38]. The effective mass of electrons was chosen to be 
ms = 9m, [39]. 

The Ohmic current density was calculated from j = oE with o = oro, = 
100 S/m. Due to the high conductivity of the TiO, matrix, the Ohmic current den- 
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Fig. 2 Comparison between tunneling current and Ohmic current density for a distance between 
electrode and filament of 0.5 nm [40] 


sity is significantly larger than the tunneling current density even in the case of the 
smallest possible distance between filament and electrode (see Fig. 2). 
Consequently, a generalized Ohm’s law, 


j=o@EQ@), (8) 


with the local conductivity o (r) of the respective material and the local electric field 
E(r) is used to describe the electron current density. Since the dynamics of the system 
is quasi-stationary, the electric field can be written as E = —V®. The differential 
equation for the electrostatic potential is thus given by 


V -(a(r)V) =0 (9) 


To solve this differential equation, Dirichlet boundary conditions are applied to the 
upper and lower interfaces. Periodic boundary conditions are applied to all other 
interfaces. The differential equation is solved numerically using the successive over- 
relaxation method on a structured grid. The current J through the device is calculated 
using the integral 


r= f ope) nda. (10) 
A 


This can be evaluated at any vertical position, since the continuity equation is implic- 
itly satisfied everywhere. 

All rates for the chemical and physical processes in the ECM cell are exponen- 
tially dependent on the local temperature. The temperature is therefore a decisive 
parameter for the memristive behavior of ECM cells. Due to the fact that in the LRS 
a metallic filament forms through the high resistance electrolyte matrix and connects 
the top and bottom electrodes, the current flow through the ECM cell is concen- 
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trated to the filament. This leads to a significant current density through the filament 
and thus potentially to a large temperature due to Joule heating. Consequently, the 
assumption of a constant temperature may not be justified and therefore the tem- 
perature in the device is calculated. The material dependent parameters of the heat 
conduction equation are also listed in Table 1. To investigate the effect of temperature 
on the memristive behavior, the temperature in two different materials, Ag and TiO,, 
is calculated. The time constants of temperature evolution in these materials can be 
estimated based on a scale analysis to Tag X 5.7 x 10715 s and Tag œ 4.2 x 107! s 
with a typical length scale of the system of 1 nm. The typical time scale of ion move- 
ment accordingly results from the largest jump rate k;; of the ions of the respective 
iteration, 


Th 6 nie Ea x 10725 (11) 
PO kij 9 kT i 


with the uniformly distributed random number ¢ €]0, 1]. The timescale of memris- 
tive behavior is thus significantly larger than the time constant of the temperature 
development. Therefore, the steady-state heat equation, 


-V . (AVT) =j- E, (12) 


is solved to calculate the temperature. A(1r) is the material-dependent thermal con- 
ductivity, whereas the right hand side represents the Joule heating source term. The 
upper and lower boundary surfaces are set to room temperature throughout the sim- 
ulation. Periodic boundary conditions are again used at all other interfaces. This 
differential equation is solved numerically on the grid, which was also used for the 
potential. To account for the influence of temperature on the ion motion, the cal- 
culated temperature is inserted into the rate equations of the chemical and physical 
process rates. 


4 Results and Discussion 


The simulation model was used to calculate the IV characteristics of the Ag/TiO,,/Pt 
ECM cell as well as the atomic state and switching kinetics. An ideal voltage source 
was applied to the Ag electrode, whereas the inert Pt electrode was set to the constant 
potential of 0 V. The source voltage and the voltage applied to the device, as well as 
the calculated IV characteristic are shown in Fig. 3. The calculated IV characteristics 
is in excellent agreement with experimental measurements as reported by Yang [41]. 
Five instances of time characteristic of the applied voltage are indicated by the 
numbers (1)—(5) and six different instances of time characteristic of the reset process 
are indicated by the letters (a)—-(f). 

The voltage ramp is applied as follows: The voltage source initially applies a 
voltage ramp with the slope 0.5 V/s to the device. To prevent an electrical breakdown, 
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the current is limited to a maximum value of Icc = 50 pA. If the current through the 
device exceeds this value, the voltage source is replaced by an ideal current source 
which provides exactly Icc. Here, the linear relationship between current and voltage 
through the device can be exploited. Using a test voltage and a corresponding test 
current, the resistance of the device, Rgcm, can be obtained. From this resistance 
and the current Zcc it is possible to calculate Vcc = Recm/cc, i.e., the actual voltage 
applied to the device. This applied voltage differs from the source voltage. 

When the (original) source voltage reaches 0.5 V (this value was chosen to ensure 
resistive switching), the slope of the voltage ramp is inverted to a value of —0.5 V/s. 
If as a result the current drops below Zcc, the current source is again replaced by a 
voltage source. When the voltage reaches —0.25 V, the slope of the voltage ramp is 
reversed to 0.5 V/s until the voltage returns to 0 V. 

The calculated temperature distribution (right) and the corresponding atomic state 
(left) for the five selected instances of time (1)—(5) from Fig. 3 are depicted in Fig. 4. 
In the original state, where the applied voltage is 0 V, no current flows through the 
device. All Ag atoms are within the electrode [time (1)]. Since no current flows 
through the device, the temperature in the device is at room temperature everywhere. 
When a positive voltage is applied to the Ag electrode, the oxidation process starts 
at the electrode due to the electric field induced by the externally applied voltage. 
The oxidized, positive Ag? ions move through the solid-state electrolyte, forming a 


Fig. 3 Input/output behavior 
of the ECM cell. Top: : — Applied Voltage 
Ramped source voltage (red) $ 
and voltage applied to the ' 

simulation region (blue) as a 
function of time. Bottom: IV 
characteristics plotted versus 
source voltage. Five 
important instances of time 
during the voltage ramp 
(green dots) and six 
important instances of time 
during the reset process (red 
dots) are marked. Reprinted 
from [14], with the 
permission of AIP 
Publishing 
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Fig. 4 Status of ECM cell shown for the five marked instances of time from Fig. 3. Left: Status 
of filament growth. Right: Corresponding temperature distribution. Reprinted from [14], with the 
permission of AIP Publishing 
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Fig. 5 Calculated current density distribution through the simulation area at time (2) from Fig. 3 
immediately after electroforming of the conductive filament. Reprinted from [14], with the per- 
mission of AIP Publishing 


stable nucleus at the Pt electrode. They reduce at this nucleus and form a conductive 
filament through the solid-state electrolyte. Once the conductive filament connects 
the two electrodes, a significant current flow through the filament is observed [time 
(2)]. 

At this point it is important to mention that resistive switching does not neces- 
sarily require a connection between the two opposing electrodes. In addition to a 
connection, it is also possible for a gap to remain between the filament and the Ag 
electrode. Either a tunneling current flows across this gap, if the resistance of the 
solid electrolyte is correspondingly high, or an Ohmic current, as presented here. In 
both cases, the current through the device is significantly smaller than in the case 
of a connection through the filament, which leads to a lower temperature inside the 
device for the same applied voltage [42]. Therefore, the retention of a gap can only 
be guaranteed by a correspondingly small current limit. 

Figure 5 shows the current density shortly after the filament has connected the 
upper and the lower electrode [time (2)]. This current density is used as a source term 
for the temperature calculation. As expected, the current mainly flows through the 
conductive filament and leads to a local heating of the device. The temperature of 
the device increases to a maximum value of 320.9 K due to Joule heating. When the 
current reaches the limiting current Jcc, the voltage applied to the device decreases 
due to the reduced resistance (blue line on the right side of Fig. 3). Due to this reduced 
voltage during current limiting, further growth of the filament is significantly slowed 
down. 

If the voltage polarity is reversed, the current increases again [time (3)]. Therefore, 
the temperature in the device also rises again and reaches values around 305 K. 
As soon as the filament breaks and thus the connection between the two opposing 
electrodes is dissolved [time (4)], the temperature drops again to room temperature. 
Due to the electric field within the solid state electrolyte, the filament is degraded 
[time (5)]. 

Since no nucleation seed was set in this simulation, nucleation occurs at random 
positions of the Pt electrode. At these nucleation sites, reduction occurs preferentially 


170 S. Dirkmann et al. 


E (V/m) 


1.5-10ë 
1.0-10ë 


5.0-107 


Fig. 6 Calculated magnitude of the electric field in the simulation area at time point t = 0.66s 
directly before electroforming of the conductive filament. Reprinted from [14], with the permission 
of AIP Publishing 


and stable clusters are formed. Figure 6 shows the magnitude of the electric field at 
time t = 0.66s. At the positions of the clusters, the electric field is increased due 
to sharp edges and because of a reduced distance between cluster and Ag electrode. 
Therefore, the growth of the filament is strongly accelerated at these positions. Due 
to the influence of the inhomogeneous electric field within the solid electrolyte, large 
clusters grow faster than small ones and eventually form a conductive filament. 

Due to the periodic boundary conditions, a copy of the simulation box can be 
imagined attached to the simulation box in x and y direction for the electric field 
calculation. The electric field in these copies of the simulation box influences the 
electric field in the simulation box. This is correct in so far, since the real device 
extends much further in x and y direction than the simulation box. Since the surface 
roughness caused by the inhomogeneous distribution of the Ag in the simulation 
box is randomly distributed, periodic boundary conditions are a good assumption. It 
limits the interpretability to distinguish single or multi filament behavior, however, 
as coupling may occur. 

Due to the fact that filament growth strongly depends on the electric field, the set 
time varies with the applied voltage. To investigate the switching kinetics of the ECM 
cell, different constant voltages were applied to the device and the time until electro- 
forming of a filament was calculated. Figure 7 shows the result of these calculations. 
With set times in the range of 0.1 ms to 10s depending on the applied voltage, the 
calculated switching kinetics of the device are comparable to corresponding experi- 
mental measurements of ECM cells [43]. 

Another aspect of particular interest in the context of the presented simulations 
are the kinetics of the reset process. At the time of maximum negative current, the 
maximum temperature is 304.6 K. Even though higher temperatures are expected for 
higher power densities within the device, this simulation result means that a critical 
temperature rise is not an issue for the typical use of ECM cells in integrated circuits 
[44]. Thus, it is clear that the dissolution and filament reset process is predominantly 
caused by the electric field. 
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Unlike the set process, which is relatively abrupt, the reset process is often gradual. 
In addition, the reset process is stochastic and varies from cycle to cycle [45]. Figure 8 
shows the atomic state of the conductive filament for the six instances of time of the 
reset process (a)-(f) selected in Fig. 3. This figure is instructive to explain the reset 
process in detail. At time (a), the filament is completely established and the reset 
process has not yet started. Due to the electric field, Ag atoms of the filament oxidize 
at random positions depending on the potential drop between filament and solid 
electrolyte and move away from the filament. 

Figure 9 shows the distribution of the electrostatic potential over a cross-section 
of the ECM cell at time t = 2.2 s, shortly before the conductive filament breaks. Due 
to the connection through the conductive filament, the potential is nearly linear from 
the top to the bottom electrode. A slight variation is observed as the magnitude of the 
electric field increases at constrictions of the filament. The filament is subject to the 
electric field approximately uniformly in position and provides oxidation processes. 
The resulting thinning of the filament leads to several weak connections between the 
upper and lower electrode [time (b)], shown by red circles in Fig. 8. In consequence, 
the resistance of the device grows successively, subject to the random dissolution 
and ion motion. This leads to the stochastic behavior of the reset process. At these 
weak connection spots, both the temperature and the electric field increase, due to 
the enhanced current density and resistance. The breakup of the conductive filament 
occurs preferentially at these narrow spots [time (c)]. After the filament has split. 
It can nevertheless reconnect [time (d)], which again leads to a drop in resistance 
and thus to an increase in current. The final breakup of the conductive filament is 
shown at time (e). As the filament recedes, isolated islands are formed. Additionally, 
it can be observed that the growth of the filament also starts at the active Ag elec- 
trode. This growth mode can also be shown in other simulations [21] as well as in 
experiments [46]. 

At this point, two more important points should be discussed. First, it is important 
to note that the presented simulation model is only valid if the conductivity of the 
electrolyte matrix is such that Ohmic current dominates over tunneling currents and 
ionic current conduction. If this is not the case, these conduction mechanisms would 
have to be accounted for in the continuity equation. 
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Fig. 8 Atomic configuration of the conductive filament at the six times marked in Fig. 3. Marked 
instances of time of the reset process. Reprinted from [14], with the permission of AIP Publishing 
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Fig. 9 Potential distribution at time t = 2.2 s directly before electroforming of the conductive 


filament in a section through the simulation area at the position of the filament. Reprinted from 
[14], with the permission of AIP Publishing 
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Second, it should be pointed out that this model focuses especially on the influ- 
ence of temperature and electric field on the reset process. Another mechanism of 
electromigration was considered in this model only indirectly, through the process 
of re-oxidation. In the literature, this mechanism is hardly discussed in the context 
of the reset process of ECM cells [47, 48]. Considering electric fields in a good 
electrical conductor, the force on an ion can be expressed as a function of an effec- 
tive charge number. For silver, the effective charge number is —26 [49]. Thus, the 
force of the electron flow dominates over the force of the electric field on Ag in the 
filament. It becomes particularly large (but disappears in quantum point contacts) at 
the narrow spots of the filament. Electromigration can thus facilitate movement of 
Ag out of the filament. Once they dissolve from the filament, they have a monovalent 
charge and are no longer affected by electromigration, but by the electric field, since 
the electron current is confined to the filament according to Fig. 5. Consequently, 
electromigration may be a crucial mechanism for the initial breakup of the filament. 
Especially due to the magnitude of the force on Ag in the filament, this mechanism 
should be considered in future extensions of this model. 


5 Conclusions and remarks 


In this chapter, the memristive behavior of ECM cells was investigated using a typical 
Ag/TiO,/Pt ECM cell as an example. The focus was on the phenomenon of resistive 
switching, the physical processes of filament growth, and the driving forces of the 
reset process. A simulation model was developed to address these aspects. The ion 
motion was described using the KMC method. The continuity equation assuming 
purely Ohmic behavior was solved to calculate the current density and the electric 
field. Additionally, the steady-state heat equation was solved to calculate the temper- 
ature within the device, subject to Joule heating. First, it was shown that the tunnelling 
currents are negligible compared to Ohmic currents in the devices at all times. The 
formation and dissolution of a conductive Ag filament could be identified as the main 
cause of resistive switching. In addition, the main chemical and physical processes 
leading to the filament growth could be described, such as oxidation, diffusion, reduc- 
tion, and nucleation. It was shown that the main driving force for filament growth 
is the electric field. A large electric field results when the distance between filament 
and electrode becomes very small. Therefore, growth occurs predominantly at the tip 
of pre-formed (incomplete) filaments. Furthermore, it was argued that the calculated 
IV characteristics of the device is in very good agreement with measurements. 

To investigate the different physical mechanisms that lead to the reset process, 
the distribution of the current density in the device and the resulting increase in 
temperature were calculated. Accordingly, the maximum current density is about 
8 x 10! A/m? and, as expected, flows mainly through the conductive filament. The 
maximum calculated increase in temperature during the reset process is approxi- 
mately 5K. Accordingly, it was concluded that the main driving force of the reset 
process is the electric field, not the temperature. In addition, it was discussed that 
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the force due to electromigration on Ag in the closed filament may not be negligible. 
Up to now, this process has only been indirectly included in the simulation by the 
process of re-oxidation from the filament. Finally, the reset process of the atomic 
scale was described in detail. 
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Abstract The two main features of the memristive devices which makes them the 
promising candidates for neuromorphic applications are low power consumption and 
CMOS compatibility. The monolithic integration of memristive devices with CMOS 
circuitry paves the way for in-memory computing. This chapter focuses on the factors 
governing the CMOS integration process. Firstly, the influence of CMOS baseline 
technology selection on the memristor module is briefly discussed. Secondly, the 
selection of metal level interconnects and their effect on the memristive device per- 
formance is explained. Further, the widely used deposition technique for the CMOS 
compatible memristive switching layers is presented. Finally, the implementation of 
the optimized process for the fabrication of the memristive module and its influence 
on the device performance is presented in terms of electrical characterization results. 
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1 Introduction 


The well know and widely used contemporary computer hardware architectures 
namely, von Neumann, Harvard, Graphics Processing units (GPU) etc.have their 
memory and logic fabricated in different process nodes [1, 2]. The scalability gap 
existing between logic and memory components is the main reason for their fabrica- 
tion divergence. Such hardware configuration with separate memory and logic leads 
to a latency delay in moving the data between the two, which is called as von Neu- 
mann bottleneck (VNB) [3]. As a result, the throughput of the hardware architectures 
becomes limited and several approaches had been proposed to overcome this issue. 
However, a change of the hardware architecture become unavoidable to solve the 
VNB problem. Memristive devices can be easily fabricated in the complementary 
metal-oxide semiconductor (CMOS) baseline technologies which are used for the 
logic circuitry. This possibility of monolithic integration of memory and logic pave 
the way for the future in-memory computing hardware architectures [2]. 

Moreover, memristive devices are considered as one of the potential candidates in 
the field of neuromorphic and edge computing applications due to their fast switching, 
multi-level conduction, low power consumption, high scalability and CMOS compat- 
ibility [4]. In order to obtain a reliable and reproducible performance of memristive 
devices, integrating them with the CMOS transistors is very essential. Firstly, the 
transistor which is fabricated in series with the memristor, prevents the sneak path 
currents in case of memristive arrays by acting as a selector device [5]. Secondly, 
the transistor limits the current through the memristive devices by setting the current 
compliance with a specific gate voltage bias [6], thus preventing the hard breakdown 
of the devices. Finally, the integration of memristive devices with CMOS platform 
reduces undesired parasitics [7]. 


2 General Technological Aspects of Integrated Memristive 
Devices 


In order to integrate the memristive module into the CMOS baseline technology, the 
below aspects need to be considered from the technology point of view: 

e CMOS baseline technology node. 

e Metal level selection in back-end-of-line (BEOL) interconnects. 

e CMOS compatible memristive switching layer. 

e Optimization of memristor module fabrication process steps. 
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2.1 CMOS Baseline Technology Node 


The first step towards integrating the memristive devices with CMOS transistors is to 
choose a compatible CMOS baseline technology. The smaller the technology nodes, 
the lower the power consumption and the faster the resistive switching [8]. The size 
of the memristive devices integrated in the BEOL process can be scaled down to 
the size of 10 x 10 nm? [9]. However, scaling down the lateral dimension of the 
CMOS transistors is a challenge, due to the voltage levels required to perform the 
forming and reset operation [10, 11]. It is in the best interest of the memristive device 
applications to scale down the transistor size while maintaining a memory window of 
at least 10 [9]. In this work, the small transistors of gate length 130 nm and gate width 
150 nm from the 130 nm CMOS baseline technology of IHP is chosen for memristive 
module integration. Figure la shows the transmission electron microscopy (TEM) 
cross section with energy dispersive X-ray (EDX) analysis of the 1T-1R integrated 
test structure. 

In general, the memristive module consists of a metal-insulator-metal (MIM) stack 
as shown in Fig. la. The memristor module is integrated by using only one additional 
MIM mask as shown in the Fig. 1b. The dimensions of the via which comes on top of 
the memristive device in the BEOL interconnects, governs its dimensions. In general, 
the memristive device size has to be larger than the size of the via as shown in Fig. 1b. 
The contact to the top electrode of the memristive device is made through this via. 
Additionally, a predefined layout design rule ‘enclosure’ margin of memristive layer 
(MIM layer) with respect to the metal layer beneath (Enc.1 in Fig. 1b) and the via 
layer above (Enc.2 in Fig. 1b) has to be maintained. This enclosure margins are aimed 
to prevent possible dry etch related damages, obtain reliable contacts between the 
metal layers and to achieve reproducible process parameters over the entire wafer. 
The smallest possible size a memristor could take is the size of the via. In order to 


Fig. 1 a TEM cross section with EDX based chemical composition maps of memristive module 
integrated into a 130 nm CMOS technology of IHP and b topview layout design rules for the 
memristive module 
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reduce the size of the memristive device, intensive process developments are going 
on to fabricate the device on top of the via [12]. However, this approach of fabrication 
is out of the scope of this work. 


2.2 Metal Level Selection in BEOL Interconnects 


In order to have a better control over the performance of the memristive devices, 
it is important to fabricate them close to the transistor. However, it should not be 
placed so close that it affects the performance of the transistor. The memristor is a 
two terminal device and, in principle, the module could be placed between any two 
metal layers in the BEOL interconnects. We avoid to place the cell on Metall in 
order to maintain the normal functionality of the CMOS transistors and to reduce the 
undesired parasitic and substrate capacitance [13, 14]. The selection of the BEOL 
metal level for the integration of the memrsitive devices depends on the technology 
used for fabrication, field of application and the material used for the memristive 
switching layers [15-18]. Additionally, the desired thermal budget is one of the 
reasons behind choosing the specific metal level for the memristor module integration 
[19]. Further, the dimensions of the memristor devices and the allowed size of the 
via on top of the device according to the defined design rules in CMOS baseline 
technology also plays a role. 

The BEOL thermal budget mainly consists of the thermal energy transferred to 
the wafer from the fabrication process steps including metallization, via-interconnect 
and inter layer dielectric (ILD) depositions. Additionally, the thermal budget consists 
of a forming gas passivation step in a diffusion anneal furnace towards the end of line 
(EOL) fabrication process at BEOL compatible temperature. The passivation step 
neutralizes the dangling bonds present at the interfaces and improves the contact 
between silicon and metal and thereby improving the electrical properties of the 
fabricated devices [20]. The thermal energy encountered by the memristor module 
is chosen carefully through the selection of the metal layer on which the module is 
fabricated [12]. The BEOL thermal budget steps activate the properties of the oxygen 
scavenging layer (OSL) present in the memristor module and thereby reduces the 
oxygen content in the memristor switching layer [21]. 

The thermal budget plays a significant role in selecting the phase of the memristive 
switching layer. The phase of the memristive layer illustrates the distribution of the 
oxygen vacancies and in turn the performance of the devices [22]. In polycrystalline 
layers, the oxygen vacancies are concentrated mainly at the grain boundaries whereas, 
they are homogenously distributed in amorphous layers [23]. The grain boundaries 
present in the polycrystalline memristive films results in higher device-to-device 
(D2D) variability. Whereas, the amorphous memristive films results in reduced D2D 
variability [24, 25]. 
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Fig. 2. XRD of as-deposited and sinter annealed HfO2 layers 


Fig. 3 XRD of as-deposited 
and sinter annealed Al doped 
HfOp layers 
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It can be clearly seen from the X-ray diffraction (XRD) patterns shown in Figs. 2 
and 3 that the as-deposited layers of HfO2 and Al doped HfO, are amorphous in 
nature. Both the layers were annealed at 400 °C for 90 m which is the BEOL thermal 
budget in 130 nm CMOS technology of IHP. The HfO> layers turns polycrystalline, 
whereas the Al doped HfO, stays amorphous after the annealing process. 
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2.3 CMOS Compatible Memristive Switching Layer 


The successful integration of the memristive module into the CMOS baseline tech- 
nology requires a CMOS compatible memristive switching layer. Several CMOS 
compatible memristive layers namely, TiO? [26], AlzO3 [27], SiO2 [28], ZrO» [29], 
TazOs [30], HfO2 [31] etc. have been studied in the literature for various applications. 
Among others, HfO> is widely studied and used as a memristive switching layer due 
to its CMOS compatibility, simple precursor chemistries used for the deposition pro- 
cess, high dielectric constant and well developed deposition processes [32]. Along 
with the selection of the appropriate top and the bottom electrodes and their areas, 
the memristor stack configuration decides the type of switching behavior, namely, 
interface based switching or filamentary based switching [33-35]. The former is out 
of scope of this work. 

Atomic layer deposition (ALD) is one of the prevalent methods used for the 
deposition of HfO2 memristive layers. The deposition temperature is compatible 
with the BEOL thermal budget. The ALD process produces stoichiometric, highly 
uniform and conformal layers. The soft breakdown in a fully stoichiometric HfO 
layer for the first time to create a conductive filament (CF) by using an inert metal 
electrode like TiN is almost impossible [36].Hence, an active metal electrode like Ti 
which has a high affinity towards oxygen needs to be deposited on top of the ALD 
HfO, layer. The Ti metal electrode acts as an oxygen scavenging layer (OSL) which 
scavenges the oxygen from HfO, layer during the electro-forming operation and 
thus enables the creation of a CF [37]. The CF can be disrupted and formed multiple 
times depending on the direction of the electric field applied. This determines the 
high resistance state (HRS) and low resistance state (LRS) of the device, respectively. 

The HfO2 memristive layer used in this work is deposited by using thermal atomic 
layer deposition (TALD) process at 300°C in the Pulsar® module from ASM. The 
layers are grown by using a self-terminating reaction technique. The deposition takes 
place by alternate pulsing of HfCl4 and H20 precursors. Halide based precursors are 
preferred over the metal organic based precursors in order to reduce the impact 
of carbon atoms on the performance of the memristive devices [19]. Further, it is 
possible to incorporate the dopants into HfO, layers by replacing the Hf pulse with 
the dopant pulses at regular intervals. The doping percentage could be altered with the 
number of dopant pulses. Mostly, HfO2 memristive layers are doped with Al atoms 
in order to keep the memristive layers in amorphous state due to the aforementioned 
benefits. 

Finally, HfO2 and Al2O0; layers could also be deposited in a stack without vacuum 
breakage. Initially, the Al,O3 layers of thickness | and 2 nm are deposited on the TiN 
metal layer by using Trimethylaluminum (TMA) and H20 as precursors at 300°C. 
The deposition of Al,O3 is immediately followed by the HfO2 deposition without 
vacuum breakage. The TEM cross section with EDX based chemical composition 
maps clearly shows the difference between the memristive layers with and without 
Al O; layers in Fig.4. Substantial research is going on in the area of HfO2 /Al,03 
bi-layer memristive devices. Addition of a thin Al20; layer below the HfO, layer has 
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Fig. 4 EDX based chemical compositional maps of the memristor layers a HfO2 b HfO2:A1203 
(8 nm: 1 nm) c HfO2:Al203 (8 nm: 2 nm) 


demonstrated an improved performance of the memristive devices in terms of HRS 
reliability [15, 38]. Further, the analog behavior is observed in the bi-layer mem- 
ristive devices which improved the accuracy of pattern recognition in neuromorphic 
computing systems [39]. 


2.4 Optimization of Memristor Module Fabrication Process 
Steps 


In spite of their low power consumption, fast switching, CMOS compatibility and 
many other potential features, the D2D and cycle-to-cycle (C2C) variability, relia- 
bility and performance of the memristive devices still remain a challenge [40]. Many 
approaches with respect to electrical characterization and fabrication technology, 
have been studied in the literature to reduce the variability and improve the per- 
formance of the devices. One of the ways is to program the memristive devices by 
using write and verify schemes like the incremental step pulse with verify algorithm 
(ISPVA) [41]. The small incremental voltage steps with read current operation in 
between allow the scheme to control the formation/disruption of the CF and hence 
result in reduced variability and supports the multi-bit operation [5]. Further, in terms 
of technology, the carbon content in the memristive layers is reduced by using inor- 
ganic precursors to obtain the promising results in case of D2D and C2C variabilities 
[19]. 

One of the ways to improve the memristive device performance from technology 
point of view is to optimize the device fabrication technique in the BEOL integration 
process. In this work, we use the TiN/HfO2/Ti/TiN memristive devices fabricated 
in the SiGe-BiCMOS technology of IHP to illustrate the three different fabrication 
approaches type 1, 2 and 3 and their impact on the electrical performance of the 
devices [42]. The memristor device stack consists of 150 nm sputter deposited TiN 
top electrode (TE) and bottom electrode (BE), 8 nm ALD deposited HfO, layer and 
7 nm sputter deposited Ti on top of memristive HfO, layer. The reactive-ion etching 
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Fig.5 Process flow of the memristive devices fabricated using three different approaches. Reprinted 
from [42] (License ID: 1252381-1; https://marketplace.copyright.com/) 


(RIE) of the stack is one of the crucial steps in the memristive module integration. 
The RIE etching of the stack is challenging owing to the fact that, the stack consists 
of the Ti metal layer which could get oxidized during the subsequent process steps. 
Additionally, the etch residuals from the top electrode if prevail on the sidewalls of 
the devices degrade the performance of memristive devices [43, 44]. 

The process flow of the devices fabricated using three different fabrication 
approaches type 1, 2 and 3 are shown in the Fig.5. In case of type 1 approach, 
the Ti/TiN and HfOz layers are RIE etched in one single step by using photoresist 
as the mask. In case of type 2 approach, only the Ti/TiN layers are etched by using 
photoresist as the mask. The HfO, layer is etched by using TiN as the hard mask. 
Type 3 approach follows similar process flow as type 2 until the Ti/TiN TE etch. 
However, the Ti/TiN layers in type 3 approach are covered with Si,N,O, spacers on 
the sidewalls before the HfO2 memristor layer is etched. Finally, the devices fabri- 
cated in all the three approaches are encapsulated by depositing Si, N,O, layers. The 
RIE etch steps in all the three fabrication approaches are followed by a plasma ashing 
step and a wet etch step called neutral oxide etch (NOE) which removes the etching 
residuals and polymer impurities from the wafer surfaces, respectively. Further, in 
case of type 1 approach, due to the patterning of HfO2 layer using photoresist, the 
residuals of the resist prevail even after the ash and NOE clean steps. Hence, an 
oxygen RIE plasma step is carried out after the memristor layer etch, which removes 
the residual resist and thereby assists the subsequent ash and NOE clean steps [45]. 

The performance of the memristor devices is tested initially on the dedicated 
process control monitor (PCM) structure as shown in Fig.6a. The PCM structure 
consists of 2457 memristor devices of size 10 x 10 um? connected in parallel. 
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Fig. 6 The pristine state currents measured on a a dedicated PCM structure of memristive module 
fabricated using b type 1 c type 2 and d type 3 approaches. Reprinted from [42] (License ID: 
1252381-1; https://marketplace.copyright.com/) 


The structure demonstrates sensitive device topologies with 9828 corners and 98,280 
um of perimeter. Further, the quality of the memristor devices, edge patterning and 
their electrical performance could be checked from the in-line tester measurements 
inside fab. In case of out-of-spec electrical results, the memristor fabrication process 
steps could be reworked. 

The pristine state currents of the memristive devices can be considered as a direct 
manifestation of their switching performance. The wafer-scale level pristine state 
currents of the memristive devices fabricated using three different approaches are as 
shown in the Fig.6. The pristine state currents are measured at a voltage of 0.1 V. 
It is clearly shown that the devices fabricated in type 3 approach using the spacers 
and encapsulation techniques exhibit lower pristine state currents compared to the 
memristive devices fabricated in type | and 2 approach [42]. 

The equivalent circuit of a memristor always consists of a resistor and a capacitor 
in parallel, to account for the parasitic losses [7]. The parallel resistor in this case is 
the side walls of the memristor device. In contrast to type 1 and 2 devices, the type 3 
devices have their side walls of the TE protected by Si,N,O, spacers before etching 
the HfO, layer. Due to this, the possibility of Ti containing polymers reaching the 
side walls of the memristive devices is considerably reduced. Hence, the value of 
parallel resistance is kept high which eventually results in low leakage current values 
in type 3 devices [42]. 
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Fig.7 TEM cross section with EDX analysis of the side walls of the memristive devices fabricated 
using a type 1, b type 2 and ¢ type 3 approaches. Reprinted from [42] (License ID: 1252381-1; 
https://marketplace.copyright.com/) 


Fig. 8 The MW extracted 
from the 1'¢ and 50‘h set 
cycles of memristive devices 
fabricated using three 
different approaches. 
Reprinted from [42] (License 
ID: 1252381-1; https:// 
marketplace.copyright.com/) 
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Further, to analyze the effect of three different fabrication approaches on the 
elemental composition of the memristive stack, TEM cross section with EDX based 
chemical composition analysis is performed (Fig. 7). Memristive devices of size 75 
x 75 um? are chosen for the analysis. In case of type 1 devices, Ti is oxidized at the 
side walls and there is a formation of Ti,O,N, on the top and bottom electrodes. The 
oxygen RIE plasma step which assists the removal of residual resist, might possibly 
have caused this oxidation. The type 2 devices show the formation of Ti,O,N, mainly 
on the TE, while it is considerably reduced in case of type 3 devices. Differentiating 
between the spacer and the encapsulation layers is hard due to the usage of the 
same material and the RIE etching angles. Even though, it is hard to notice the Ti 
impurities on the side walls of the devices, the electrical measurement results support 
the claim [42]. 

The 75 x 75 um? memristor devices are further characterized for their resistive 
switching operation. The type 3 devices exhibited less variability in terms of forming, 
reset and set voltages. The devices are cycled for 50 cycles of reset and set operations. 
The memory window (MW) of the devices is extracted from the 50 set cycles. It can 
be clearly noticed from the Fig.8 that the MW decreases with cycling in type 1 
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devices while it increases in type 2 and 3 devices. Further, the type 3 devices show 
slightly higher MW with less variability compared to type | and 2 devices [42]. 


3 Conclusion 


The necessity and importance of memristive device integration into a CMOS baseline 
technology is discussed. The technological aspects of memristive module integration 
are presented under four different sections: (1) CMOS baseline technology (2) Metal 
level selection in BEOL interconnects (3) CMOS compatible memristive switch- 
ing layer (4) Optimization of memristor module fabrication process steps. Under 
the first section, the factors governing the size of the memristive devices and, the 
layout design rules for the memristive module integration are briefly discussed. In 
the second section, the metal level selection in terms of CMOS performance and 
thermal budget of the BEOL processes is presented. Additionally, the phase of the 
memristive switching layers and their influence on the performance of the devices 
in terms of D2D and C2C variability is presented. In the third section, the ALD and 
the reason behind its wide usage as a memristive layer deposition process is briefly 
discussed. In the fourth section, the memristor module is fabricated using three differ- 
ent approaches. The optimized fabrication process steps are studied systematically in 
terms of TEM cross section with EDX based chemical analysis and correlated with 
the device performances. The devices fabricated using type 3 approach with spacer 
and encapsulation technique exhibited lower pristine state currents on a wafer scale 
analysis. Finally, the extracted MW of type 3 devices exhibited higher values with 
slightly less variability in comparison with type | and type 2 devices. 
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A Wave Digital Approach Towards A) 
Bio-inspired Computing Using giecik 
Memristive Networks 


Dennis Michaelis and Karlheinz Ochs 


Abstract Bio-inspired computing is a promising approach to tackle problems that 
are too complex for state-of-the-art computing approaches. It is a highly interdisci- 
plinary field of research as it requires expertise, which ranges from understanding 
biological organisms (psychology, physiology zoology, biology, bio-chemistry) to 
building electrical circuits (electrical engineering, physics). This chapter focuses on 
theoretical circuits designs and concepts to abstractly map biological behavior to ide- 
alized circuits. We utilize a special digital emulation technique as a tool to digitally 
replicate circuits and hence bridge the gap from biological behavioral observation to 
hardware circuit designs. Application examples include parameter optimization of 
a neuronal oscillator, gait pattern generation, multi-neuron communication, neuro- 
plasticity and optical illusions. The fundamental circuit elemental to realize several 
bio-inspired circuit designs is the memristor, which is essentially a resistor with a 
memory. Since memristors are hard to fabricate with the current state of technology, 
digital emulators are a useful tool to accelerate development cycles and investigate 
the circuits of the next generation. 


Keywords Memristor - Electrical circuits - Emulation + Neuromorphic 
engineering - Self-organization 


1 Introduction 


It is worth noting that a digital emulator is different compared to a hardware emu- 
lator and naturally comes with some benefits, although it is generally harder to 
construct. Hardware emulators use electrical devices to emulate the behavior of a 
desired electrical system. Due to the intrinsic computational parallelism in a signal 
processing sense of voltage and current in an electrical circuit, such an emulation 
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circuit is a powerful instrument. However, such emulators are often hardwired and a 
change of parameters in the system of interest requires new designing of the electrical 
hardware [1, 2]. This makes hardware emulators somewhat inflexible, as a physi- 
cal exchange of electrical devices in the emulation circuit is required, sometimes 
the circuit even needs to be partially or completely redesigned. Software emula- 
tion on the contrary can change parameter values digitally. This enables not only 
uncomplicated and time-saving parameter changes, but also in-operando parame- 
ter manipulation, meaning that parameters can even be changed during run-time. 
Consequently, procedures such as parameter optimization and sensitivity analyses 
are possible, as demonstrated in [3]. It should however be mentioned that a digital 
emulation comes at a cost. First, appropriate interfaces with the help of analogue- 
to-digital (A/D) and digital-to-analogue converters (D/A) need to be implemented 
[4]. Second, digital emulators are typically more sophisticated to design compared 
to hardware emulators because the stability of the overall system must not be altered 
by the digital emulator which runs under finite precision arithmetic. 


2 Digital Emulation Technique 


The wave digital concept [5] takes into account all of the above and is here exploited 
as a digital emulation technique as it is known to preserve energetic properties such 
as passivity in a digital signal processing sense [6, 7]. It leads to a computationally 
massive parallel algorithm, which can be implemented on DSPs or FPGAs and ASICs 
for real-time applications in integrated circuits [8, 9]. A short overview of the wave 
digital concept is given below. 

Voltage u and current i of Fig. 1 are related to incident wave a and reflected wave b 
via an arbitrary positive constant R that is called the port resistance and the bijective 


transformation 
a 1 Riju 
l-lif] 8° 0 


Consequently, to obtain the wave digital model based on a reference circuit, one 
has to translate all electrical devices port-wise and their Kirchhoff interconnection 


Fig. 1 Definition of a port i 
with electrical quantities u, i ; monk 
(left) and wave quantities a, 
b with port resistance R ù R N 
(right) 
bo 


A Wave Digital Approach Towards Bio-inspired Computing ... 193 


Table 1 Wave digital sources 


Element Electrical circuit Wave flow graph 
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network separately in the wave digital domain and then connect them port-wise again 
subsequently. For example, resistive voltage and current sources described by 


u =e— Rọi and i= j — Gou, (2) 
with source resistance Rọ or source conductance Go and inputs e, j, respectively. 
Using the bijective transformation (1), their wave digital flow graphs are described 
by 

a=[l—ole+eob and a=[1+e]Rj +ọb, (3) 


respectively, with 9 = [Ro — R]/[Ro + R], cf. Table 1. In the case of ideal sources, 
the inner resistances vanish and hence 


Ro =0Se=-1 and Go=0Sso=1, (4) 
resulting in the simplified expression 
a=2e—b and a=2Rj+b. (5) 


Other selected nonreactive elements include the resistor, which is characterized 
by 


u=Zi & b=ea, (6) 


with ọ = [Z — R]/[Z + R]. After discretization and deploying the trapezoidal inte- 
gration rule due to its beneficial features regarding passivity and stability [6], the 
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Table 2 Selected wave digital reactive elements 


Element Electrical circuit Wave flow graph 
Capacitor i b 
ox: Be 
7 
L L—oa 
Eat 
20 
Inductor i -1Q-) 
= Lg 
2L 
T 


capacitor and the inductor become simple delay elements in the wave digital domain. 
By defining the step size T = tk — tk—1, one obtains 


b(tk) = a(tk-—ı) and b(t) = —a(tk-1), (7) 


for a capacitor and inductor, respectively, after their port resistances haven been 
chosen to R = T/[2C] and R = [2L]/T, respectively. Their corresponding wave 
digital flow graphs are shown in Table 2. 

Now that selected electrical devices and their corresponding wave flow graphs 
have been presented, their port-wise connection is discussed. Since all wave quan- 
tities are always related to their respective port resistance, only elements with the 
same port resistances can be connected immediately. While certain elements allow 
for an arbitrary (but positive) choice of the port resistance, such as the resistor, others 
do not offer this degree of freedom, such as the capacitor or inductor. To still be able 
to connect elements with different port resistances, parallel and series adaptors rep- 
resenting Kirchhoff parallel and series interconnections are utilized. The underlying 
equations describing a Kirchhoff series interconnection are 


17u=0 and i= lio, (8) 


with 1 =[1...1]’ being the all one vector of appropriate dimension. When the 
electrical quantities are replaced by wave quantities, cf. (1), the relation 


2R1 
I7R1 


b=[1-—y1"] with y= (9) 
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Table 3 Wave digital adaptors 


Element Electrical circuit Wave flow graph 
2 ; b2 az 

Four-port : iz R : Ro 
parallel connection z 

Ri : --t-4 Gg 

Uy U3 
o> ——o 
ti he Rz 


Four-port uz 
series connection 


ui u3 
o + o 
Rı 3 
igh 


emerges, where 1 is the identity matrix of suitable dimensions and R = diag} -1 Ry, 
with R,, being the port resistance of the u-th port. Likewise, a Kirchhoff parallel 
interconnection is described by 


Wix=O0 and u = luo (10) 


in the electrical domain, which yields the wave relationship 
b=[ly? -1] with y = —— (11) 


with G = R~!. The respective wave digital symbols of Kirchhoff series and parallel 
adaptors are shown in Table 3. 


3 Memristive Neuronal Oscillator 


The well-known Morris-Lecar model is a second-order model describing a neuron’s 
membrane potential behavior [10]. The structure of its electrical circuit is shown in 
Fig. 2 (top left) and the underlying equations are written as 
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du . 
CT = iapp — gca(u)[u — Eca] — zGgılu — Ex] — Gilu — Ey], (2) 


with the nonlinear resistor and memristor described by 


u — Uca 
gcalu) = Geo (4 ') , (13) 
Uca2 
dz u—Uki : u — Ux 
— = — z]|F h | ———— th = — 
dt [Zoo (u) z] K COS. ( Wyo ) ’ wi Zoo(u) oO ( Ur ) ’ 
(14) 


respectively, where ø (-) is sigmoid function, the capacitor models the changing mem- 
brane potential, E, and G; model the leakage current behavior. Eca is the voltage 
offset to achieve biologically meaningful calcium (Ca) concentration values and the 
nonlinear resistor gc,(u) described by (13) models the opening and closing behavior 
of the Ca-channels. Likewise, voltage offset Ex aims at accomplishing biologi- 
cally meaningful potassium (K) concentration values and the memristor Wx (z, u) 
described by (14) models the opening and closing behavior of K-channels [11]. The 
nonlinear resistor here models the opening and closing behavior of the Calcium 
channels, whereas the memristor accounts for opening and closing behavior of the 
Potassium channels. 

A fundamentally important phase of an action potential is the hyperpolarization 
phase, in which the membrane potential temporarily falls below the resting poten- 
tial. This is relevant for several learning mechanisms, e.g. spike-timing dependent 
plasticity [12]. The problem with the parameter set given in [11] is that for several 
values of the applied current iap, the hyperpolarization phase is missing. In order to 
possibly find a parameter set, where the hyperpolarization phase is present, a param- 
eter optimization on the basis of a wave digital emulation is conducted [3]. The 
wave digital model of the circuit is shown in Fig.2 (bottom right) and is obtained 
from the electrical circuit by a port-wise translation of its electrical components 
and Kirchhoff interconnection structures. For this reason, the structural similarities 
between the electrical circuit and the wave digital flow graph are immediately observ- 
able. There, implicit relationships between certain wave quantities are highlighted 
in orange. To deal with these, fixpoint iterations are utilized [13]. Because the wave 
digital model is real-time capable, in-operando parameter manipulations enable a 
parameter optimization during run-time. 

For the emulation results, an ideal current source is placed at the left-hand port 
to mimic the applied current i App and the resulting membrane potential is observed 
at the right-hand port. The action potential emergence as a reaction to the constant 
input iapp = 69 uA with the state-of-the-art parameters of [11] as the reference and 
in comparison with the in [3] optimized parameters is given in Fig. 3. 

The following is gathered. The membrane potential of the reference parameter set 
never fall below its resting potential ®,.¢ = —60.86 mV. The highlights that indeed 
the important hyperpolarization phase is missing. On the contrary, the optimized 
parameter cause the membrane potential to fall deeper while the resting potential 
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Fig. 2 Wave digital flow graph of the Morris-Lecar neuron model. Its structural, port-wise corre- 
spondence to the electrical circuit (top left) becomes apparent 
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Fig. 3 Constant current of the Morris-Lecar model with optimized parameters from [3] in com- 
parison to parameters as in [11] 


Pop = —49.35 mV is increased. This results in a clear hyperpolarization phase, indi- 
cated by the green areas in Fig. 3. In summary, a sophisticated in-operando parameter 
optimization has been conducted via the wave digital model, which resulted in a bio- 
logically more meaningful behavior. 


4 Stimulus-Driven Topology Formation 


4.1 Supervised Topology Formation 


Memristors are a suitable electrical device for supervised topology formation and 
first preinvestigations in this regard have been done in [14]. For an extension of 
this concept, descriptive sketches of neurons and their coupling strengths like in 
Fig.4(i) are considered. They are used especially in biology to visualize neuronal 
setups [15]. In this context, the nodes symbolize the neurons and the arrows represent 
the (unidirectional) synapses. The numbers indicate the couplings strength between 
O and 1, where the + and — sign stands for excitatory and inhibitory couplings, 
respectively. The synthesis of an electrical circuit based on these descriptions is 
challenging for multiple reasons. The probably biggest questions arise when aiming 
to realize the unidirectional synaptic coupling, as effects like propagation of voltages 
and currents in an electrical circuit are generally not unidirectional and hence are not 
free from feedback. 

In the following, it is explained how the wave digital model of Fig. 4(ii) achieves 
these tasks. To do so, the wave digital concept is here exploited as a modeling tool, 
since this process is more intuitive in the wave domain rather than the electrical 
domain as it will become apparent subsequently [16]. Lastly, the appropriate elec- 
trical circuit is obtained due to its direct correspondence with the final wave digital 
model. The model is subsequently verified in the context of a dog’s gait patterns. 
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Fig. 4 (i): Conceptual neuronal network with coupling strengths and inhibitory/excitatory coupling 
represented by negative/positive signs. (ii): Synthesized wave flow diagram 


Figure 5 shows the proposed synapse model to synthesize the scenario of Fig. 4(i). 
It consists of a four port circulator with two attached resistances R and R3 that 
account for the coupling strength and coupling type (excitatory for positive reflec- 
tion coefficients or inhibitory for negative coefficients). To explain its functioning, 
the reflection coefficients are expressed by 


R,—R 
eel, 2. (15) 


seal ee 


The special cases of the these reflection coefficients are of interest here in this context. 
Specifically it holds that 


Rı >œ } Q=1, 
R, 70 & o,=-l, 
Ru =R & Qn, =0. 


It can be seen that the whole spectrum o, € [—1, 1] is experienced and that the sign 
of the reflection coefficients is determined by whether the resistance value R,, is 
smaller or bigger than the circulation resistance R. Of special interest is the case, 
where R, = R, as this results in @,, = 0. This case is special, because it allows for 
unidirectional coupling. This can best be seen when the power flow is is calculated 


: 2 
Fa = | - d Pad ; (16) 
Ppost,in Qi 0 Ppost, out 
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Fig. 5 Left: Proposed synapse model designed as a wave digital model. Right: Its corresponding 
electrical circuit 


where 


2 2 2 2 
bire Apre Drost Anost 


Ppre,in = 4R >» Ppre,out = 4R’ Ppost,in = 4R’ Ppost,out = 4R 


and the total power consumption of the pre- and postsynaptic ports 


Ppre = Ppre,in — Ppre,out> Ppost = Ppost,in — Ppost,out- 


It becomes evident that 9; = 0 prohibits any power to flow from pre- towards post- 
synaptic neuron and vice versa for 92 = 0. Consequently, this enables unidirectional 
coupling if either one of the reflection coefficients is zero or complete decoupling 
if both are zero. With these insights it is possible to synthesize an electrical circuit 
corresponding to the scenario of Fig. 4(i) as shown in Fig. 4(ii). It becomes evident 
that the nature of the scattering parameter depiction is much more similar to the con- 
ceptual graph compared to a depiction in electrical quantities. Therefore, depending 
on the context, the wave digital concept can not only be a powerful emulation tech- 
nique but is also a versatile modeling tool by enabling perspectives that are quite 
unintuitive in the electrical domain, but are easily accessible in the wave domain. 

To verify the proper functioning of the concept presented above, a simple applica- 
tion example in the context of a dog’s gait patterns is examined. More specifically, the 
gait pattern running, pacing and trotting are investigated. To analyze these patterns, 
two subgroups S; and S, with the following features are created 


1. Every component within a subgroup is synchronized to every other component 
within the same subgroup (intra-group synchronization), 
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Fig. 6 Output signals of four interconnected, optimized Morris-Lecar neuronal oscillators mim- 
icking the gait of a dog changing from running (t < Tı) to pacing (Ti < t < T») to trotting 
(T < t < T3) to standstill (t > 73). Each color represents the motion of a dog’s leg as indicated at 
the top 


2. Every component of subgroup S; is asynchron to every component of subgroup 
S (inter-group asynchronization). 


By these categorizations, the three above mentioned gait patterns can be established 
by assigning the four legs of a dog to the subgroups S1, S2. 

For the emulation scenario, four optimized wave digital neuronal oscillators, cf. 
Sects. 3 and [3], are associated with a leg of a dog each as indicated by the top sketches 
of Fig. 6. They represent a neuron population that accounts for the movement of a leg. 
First results in this context have been published in [16]. The neuronal oscillator model 
of choice is the optimized Morris-Lecar neuron model of Fig. 2 with its parameters as 
mentioned in [3]. The sketches on top of Fig. 6 indicate intra-group synchronization 
(solid arrows) and intra-group asynchronization (dashed arrows). The intra-group 
synchronization is here realized by a bidirectional coupling with edge weights +1 
while intra-group asynchronization is achieved by bidirectional couplings with edge 
weights — 1. Although arbitrary but different initial conditions are chosen for the four 
neuron’s membrane potentials, the desired pattern is achieved after just two oscilla- 
tions and indeed represents the gait pattern of a running dog, cf. Fig.6. Att = 7), the 
edge weights are abruptly changed to the pacing gait pattern. Note that although the 
topology changes are abrupt, the transition behavior is actually smooth and steady. 
In fact, almost no changes with respect to the phases are visible during the first four 
oscillations after the switch, only the amplitude is temporarily reduced. Note that in 
the transition phase the underlying synaptic interconnection of the neurons does not 
match their relative behavior. For this reason, they interact destructively until this 
transition phase is completed. This is the case after approximately 10 oscillations 
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after t = T; = 250 ms. A similar behavior is observable when the synaptic coupling 
weights are changed once again to mimic a trotting gait pattern att = Ta = 1000 ms. 
During the transition phase, where the constellation of neuronal activity and under- 
lying synaptic interconnection is inconsistent, the amplitudes are damped. Then, 
after around 6 oscillations the relative positions of the membrane potentials start to 
change until completion after around 14 oscillations. Note that here the change in 
synaptic coupling weights are again abrupt, but the transition is still smooth. This is 
highly desirable because this translates to the dog being able to change gaits with- 
out tumbling. At t = 73 = 1625 ms, the current exciting the Morris-Lecar neurons 
is switched off and consequently a standstill is achieved after just one oscillation 
but in a smooth fashion. To summarize, a slow down process from a running to a 
still standing dog was successfully emulated, where its multiple gait patterns were 
mimicked in the process. An important detail worth mentioning is the fact that in 
this application example the hyperpolarizon phase is present at all times, which is 
observable by the action potential minima always surpassing the resting potential 
Pop = —49.35 mV. This once again highlights the improved Morris-Lecar neuron 
model [3]. 

It is also apparent that the here presented gait pattern generator is closely related to 
the synchronization of neuronal activity to form a biologically meaningful behavior. 
More general investigations on synchronization of linear(ized) circuits have been 
conducted in [17, 18] and also in the context of an Ising machine [19]. Furthermore, 
the first results in this context have sparked more sophisticated results incorporating 
axon growth in the case of gait pattern generation [20] and gait pattern classification 
[21]. 


4.2 Self-organized Topology Formation 


4.2.1 Optimal Two-Neuron Communications 


Itis desirable to investigate how information from an origin neuron can be transferred 
to a destination neuron optimally, i.e. with the least amount of delay. This problem 
can be formulated by graph theory, where nodes represent neurons and edges mimic 
synapses. Doing so, the task is comparable to solving a maze, where the entry of the 
maze accounts for the origin neuron and the exit accounts for the destination neuron. 
The here exploited analog parallelism is different from computational parallelism 
and enables the solution to the problem to outperform all known graph-theoretical 
algorithms [22]. The structure of the maze represents the structure of the neural 
network, where it is assumed that only nearest neighbor synaptic couplings exist 
and that an obstacle, e.g. a wall, means that no synaptic coupling exists between 
these specific neighboring neurons. First preinvestigations have been done in [23]. 
The concept can also be used to determine the longest rather than the shortest path, 
which is also a computationally complex problem [24]. 
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Fig. 7 Setup of the maze to mimic two-neuron communication, which consists of nodes p, and 
their interconnections C,, 


Stacking all neuron potentials „y and all synaptic currents ję in vectors g and 
J, respectively, allows for a condensed notation of the maze topology, cf. Fig. 7. The 
synapse voltages v are obtained by the respective neuron potential differences and 
can be expressed as 


v= N’ glu), (17) 


where N is the incidence matrix and voltage u is the potential difference between ori- 
gin neuron potential p, = u and destination neuron potential p, = 0. Consequently, 
by Tellegen’s theorem, the relationship between synaptic currents j and neuronal 
currents į is 


i= Nj, (18) 


where i = —e/i = efi. 

The synapse model mimics bidirectional (both switches closed) or unidirectional 
(only one switch closed) connection or no connection at all (both switches open) and 
is depicted in Fig. 8. This is because the switches decide whether a current can flow 
through the memductors W;, W,’ or not. Here, a wave digital model with physically 
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Fig. 8 Synapse model for two-neuron communication 
Table 4 Maze memristor emulation parameters 
ig = 38.74 pA Ro = 10 kQ 
uo =0.5 V Sp = —50 Hz/V 
N=1.5 Sn = —0.4 Hz/V 
6 =0.1 Up=1V 
y =3.8 Un = —0.5 V 


meaningful parameters of a RRAM-cell, developed in [25], has been utilized. This 
device distinguishes itself by its rapid switching behavior, which here is exploited 
to achieve the final state of the maze as quickly as possible. Consequently, synaptic 
currents j and voltages v are related by j = W (z, v) v, which in combination with 
the synaptic voltages and the neuronal currents, cf. (17) and (18), yields a nonlinear 
relationship between i and u 


i =e! NW(z, N g(u)) N gu), (19) 


which is numerically solvable in the wave digital domain by using fixpoint iterations 
for the implicit relationships between u, i and z [23]. By Chua’s closure theorem, 
the memristive network can be summarized by single memristor, see [26]. With 
the emulation parameters as shown in Table 4, there are two emulation scenarios 
investigated in the following. 

Scenario 1 involves a maze of size 20 x 20 as shown in Fig. 9 (top left). There are 
only three possible communication paths for the origin neuron (entry) to transmit 
information to the destination neuron (exit). The course of the current as a response 
to the input voltage ramp e is displayed to the right. Every notch represents the 
switching of all RRAM-cells along a certain path, which can be exploited as a detec- 
tion mechanism. Since here the current is highest among the path with the least total 
memristance, the possible communication paths are found in order of ascending path 
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Fig. 9 Comparison of a 20 x 20 maze with three possible paths from entry to exit, highlighted in 
orange, green and blue (top left) and a 50 x 50 maze with only a single path (bottom left). The 
course of the currents for the 20 x 20 and 50 x 50 maze are shown at the top right and bottom right, 
respectively 


length at t = 13 ms, t = 32 ms and ft = 39 ms, respectively. In scenario 2 a bigger 
maze of size 50 x 50, cf. Fig (bottom left), is investigated with the course of the cur- 
rent again illustrated to its right. A single notch indicates that there is only a single 
communication path possible. Note that the point in time, in which the communi- 
cation path is discovered by the setup, is also tf = 13 ms. This is the same amount 
of time required in the first scenario with the significantly smaller maze to find the 
shortest communication path. Remember that the first scenario requires 361 connec- 
tion elements and consequently 722 RRAM-cells, while the second scenario requires 
more than six times that many (2401 connection elements, 4802 RRAM-cells). 

This leads to the counterintuitive conclusion that the size of the maze has a negli- 
gible influence on the convergence time. This is due to the inherently massive analog 
parallelism and is the reason why computationally complex problems as the one 
investigated in this work can be solved efficiently by memristive circuits. While in 
the first scenario an input voltage e = 180 V has been utilized, in comparison to 
the second scenario, where an input voltage e = 500 V has been deployed, roughly 
three-times more voltage was required to compensate the time requirement, it is still 
remarkable that independent of the maze size, the presented setup requires an iden- 
tical convergence time to find the shortest communication path. Furthermore, due to 
the immense amount of memristive devices present in the setup, a run-time compar- 
ison between the here presented wave digital emulation and the circuit simulation 
software LTspice was investigated in [23]. Although the scenarios were different, a 
run-time advantage by the factor of 300 in favor of the wave digital emulation was 
discovered on an identical computational device. 
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4.2.2 Mimicking Neuroplasticity 


Here, one neuron of interest aims to find the optimal, e.g. shortest, communication 
pathways to all other neurons being part of the neural network. The basic concept 
was presented in [27], extended in [24] to solve the longest path problem and in 
[28] to mimic neuroplasticity. A fundamental scenario of optimal communication 
pathways with regards to multiple neurons is shown in Fig. 10. 

There, three interconnected neurons are depicted with different time delays that 
can be associated for example with their axon lengths. For instance, an action potential 
can travel from neuron M4 to neuron Mg with a time delay of t. The same holds 
true for a communication between neurons Mg and Nc. However, a communication 
between neurons Ma and Nc requires triple the time, namely 37. From the perspective 
of N4, there are two possibilities to transmit information towards Nc: First, the direct 
path, which would require a time delay of 3t. Second, the path over Vg, which would 
require 2t in total. The second choice is obviously better than the first with respect 
to time delay and should hence be used. This would make the synaptic connection 
between M4 and Ne obsolete. Because keeping this path (axon, axon terminals, 
synapse) alive requires energy, it would make sense from an energy perspective to 
let this path shut down, as indicated in Fig. 11 (right). 

The ambition of the following steps is to synthesize an electrical circuit that 
inherently incorporates the above mechanisms. First, the problem is abstractly rep- 
resented as a graph, where a neuron is mimicked by a node and a synaptic connection 
by a directed edge. The edge weights then represent the according time delay of the 
path. This is visualized by way of example as a nine neuron setup Ny,...,Nz 
in Fig. 11 (left). By doing so, the problem of finding the minimal communication 
paths from neuron M4 to all other neurons Mg, ..., Nz becomes the problem of 
finding the minimum spanning tree, where M4 is the root. Previously, non-circuit 
approaches to solve this problem exist, such as the Dijkstra, A*, Bellman-Ford or 
Floyd-Warshall algorithms [29]. Among other features, these algorithms differ in 
their access to information on the topology and costs in the graph. Apart from the 
inherent favorable convergence time [30], a self-organizing circuit solution to the 
minimum spanning tree problem is also desirable in the context of unsupervised 
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Fig. 10 A fundamental scenario what adaption and optimization means: The setup determines the 
paths of least delay self-organizingly and lets all other paths die off to increase energy efficiency 
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Fig. 11 Left: Initial neuronal network with dense synaptic connection. Right: Only synaptic con- 
nections enabling the least delay communication between neuron A and all other neurons in the 
network remain, all others have died off. The synapse between neurons € and F is formed after the 
synapse between neurons C and F was damaged 


learning, as the circuit itself discovers optimal communications structures for infor- 
mation transport. Unsupervised learning is also of interest in other domains, such 
as pattern recognition, where circuit-based solutions are applied [31]. The circuit to 
inherently find this minimum spanning tree in designed by synthesizing the neurons 
(nodes) and synapses (edges) as proposed by Fig. 12. 

There, two different types of memductors are utilized. The neurons are represented 
by the jz-indexed memductor, which is described by equations 


in = WulZp)Un, (20a) 
Wo Wy 
WN + zu [W0 — wh] 


Wp (Zu) = (20b) 


where W,,(z,,) is the memductance and z, is the inner state. Here, z,, = O indicates 
the neuron memductor is in the high conductance state WÑ and z, = 1 indicates the 
low conductance state WN. The state equation is described by 


Zu = gy) [0 Uy )o (Zu) + o(—uy)o( — zy), 


(21) 

gp) = SN [uy T up] o (üp = up) F SN [uu F un] O(—Uy, + Un), 
where o(-) is the Heaviside function, Syn is the steepness of the change from wh 
to WN and vice versa, up is the set voltage and uy is the reset voltage. The synapse 
between neuron u and neuron v is described by the double-indexed memductor 
model 
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Fig. 12 Circuit concept to mimic neuroplasticity in the neuronal network of Fig. 11 


Juv = Wruv(Zpuv) Ups (22a) 
Wo WP 


W, v(Z v) = 
oe WP + zu [We — w$] 


(22b) 


and its state equation 


Ziv = ECV) [o(p)o (Zur) F O(—vyy)o (1 = Zw)] F SRO (Zuv — 0.1) 


(23) 
gV) = Ss [vw = vp] o (Vav = Up) + Ss [-vw F va] o (—Vuv + Un), 
where Sp is the amplitude of the retention characteristic, Ss is the steepness of the 
change from we to ws and vice versa, vp is the set voltage and v, is the reset voltage. 
Additionally, in order for the circuit to function properly, it was explained in [27] in 

detail that 
We > Wi > We > Wh. (24) 
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Table 5 Simulation parameters 


We=1S8 Up =O0V Up =6V 

WÒ =1 mS vin =—1V Un = —0.74 V 

WP =1S Ss = —1.2 Hz/V Sy = — 100 kHz/V 
WN =1nS Sr =0.1 Hz/V 


must hold. By keeping the above mentioned specifics in mind, the setup of Fig. 11 
(left) is synthesized by the electrical circuit presented in Fig. 12. 

The mechanisms work as follows. Assuming all neuron memductors are initially 
in the high conductance state and all synapse memductors in the low conductance 
state, the root neuron MVA is attached to an ideal voltage source e mimicking its 
membrane potential activity. When the synapse memductors directly attached to 
the root memductor experienced enough magnetic flux, they transition to the high 
conductance state. At this point, both the root memductor and the newly reached 
neuron memductor are approximately at the the same voltage potential. Therefore, 
this neuron memductor experiences almost all of the voltage from e. Since the neuron 
memductors are oriented in a way such that they transition in the low conductance 
state when experiencing a negative voltage, i.e. during the hyperpolarization phase, 
it is exploited as a mechanism to signal that the path of minimal delay towards 
this neuron has been found. To further illustrate this procedure, the initial neuron 
population of Fig. 11 (left) is considered in a state where it has not yet adapted to its 
environment. The corresponding memristive circuit of Fig. 12 is simulated for this 
purpose with its parameters given in Table 5. The initial states of synapse and neuron 
memductors are chosen to z$ = 0.7 and zi’ = 0, respectively, and the input signal is 
given by a sequence of action potentials with U; = —0.75 V, U2 = 4 V and period 
T. = 1s. 

The behavior of all synapse memductance states are shown in Fig. 13, where all 
synapses’ states being part of the optimal synaptic structure formation are visualized 
on top and all others at the bottom. 

In particular, path M4 to Vg is found first at t1 = 0.45 s, followed by path Mg to 
Ne at t = 1.35 s, path Ng to Ne at t3 = 1.55 s, path Nc to Ng at t4 = 2.45 s, path 
Ne to Np at ts = 4.25 s, path Ne to Ny as well as path NF to Nz at tẹ = 5.45s, 
and finally path Mp to Ng at tı = 6.35 s. At this point, all neurons Mg, . . . , Nz in 
the setup can be reached from the root neuron M4. There are three things worth 
highlighting. First, notice that path E to H and path F to I are found simultaneously, 
which shows that no overwriting of previous states is required. Second, note that 
the communication paths are found in order of ascending total delay time from 
the perspective of M4. Lastly, all synapses associated with the memductance states 
shown in Fig. 13 (bottom) are not part of the optimal communication structure and 
hence must die off from the perspective of energy efficiency. The circuit identifies 
those synapses by all memductances whose states are not in the high conductance 
state. The proper interpretation is visualized by the blue communication paths in 
Fig. 11 (right). 
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Fig. 13 Simulation results of all synaptic memristors which are part of the optimal communication 
structure (top) and are not part of the optimal communication structure (bottom) 


However at tg = 9.76 s, the memductor We r is removed from the electrical circuit 
to account for some damage to the neural network, maybe induced by an accident 
or a disease. Because the path from Nc to MNF was part of the previously optimal 
communication structure, M4 can not communicate with all neurons of the neural 
network any more. Specifically, neurons MF and Nz are affected. However, the 
circuit is able to self-heal this damage. After a short period of time at tọ = 12.21 s, 
the memductor We transitions to the high conductance state, indicating that the 
previously died synapse between Mg and MNF is now created anew. Now the root 
neuron M4 can communicate again with neurons Ms and Nz, highlighted by the 
green arrow in Fig. 11 (right). Note that the circuit will always inherently find a way to 
self-heal the communication structure with the minimal amount of changes compared 
to the already established, but incomplete structure. By this circuit synthesis and the 
subsequent simulation results it was shown that a structure can be simultaneously 
energy-efficient and fault-tolerant. Both are features frequently associated with the 
human brain and typically are in a trade-off with one another, where more energy 
needs to be invested in order to achieve a redundancy, which in return guarantees the 
fault tolerance. 
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5 An Elementary Decision Problem via Optical Illusions 


Optical illusions are closely linked to associative learning. They also showcase that 
we permanently make unconscious decisions and it is for example when we encounter 
an optical illusion that we become aware of it, because something does not seem to 
be right. This is where the limits of our decision making is revealed. Associative 
learning or Hebbian learning was developed by Hebb, and is often summarized 
as “Cells that fire together, wire together” [32]. It describes a rule for change in 
synaptic coupling strength between mutually active pre- and postsynaptic neurons. In 
general, it attempts to explain associative learning, but is a very simplistic explanation 
that lacks details. For example, time delays caused by the axon are not considered. 
One famous example of associative learning is Pavlov’s dog [33]. There, it was 
observed that a dog produces saliva whenever he was given food. After a time, Pavlov 
discovered that the saliva was already produced when the lab’s assistant entered the 
room with the food. He developed the theory that the dog had learned to associate 
the assistant’s foot steps with the food and hence the foot steps caused the saliva 
production. He then tested the theory by ringing a bell shortly before the dog was 
given food. And sure enough, after repeating the process several times, a ring of the 
bell produced saliva in the dog’s mouth, even if there was no food given after the 
bell’s sound. He discovered that the two stimuli (sound of bell and presenting of food) 
both have to be presented within short period of time in order for the dog to associate 
the two events with one another. Since then, associative learning has been tried to 
replicate in electrical circuits as one branch of bio-inspired behavior. In this context, 
a simple form of associative learning is synthesized by a simple memristive network 
with neuronal oscillators [34]. Such oscillators are generally nonlinear models, cf. 
Sect. 3. A well-known, well-investigated nonlinear oscillator model is the Kuramoto 
model, which describes a set of multiple interconnected systems [35]. It is here 
utilized to model such a neuronal oscillator. The overall set Xx is described by 


u =w + 2 Ppi Kuv SINO — Gy) 


Èk Š Zu = Pp š (25) 
Ya = Sin(Py) 
where u = 1,...,m, @, is the angle of the „m-th oscillator, w is the intrinsic radian 


frequency, «,,, is a nonlinear coupling factor, z, is an output providing the neigh- 
boring subsystems with information about @, and y, is the output from which the 
oscillations can be observed. Note that the phases @,, itself are only integrated and 
the oscillation of the system is only due to the nonlinear output function y,. To 
incorporate the Hebbian learning rule, the coupling strength x,,, between oscillator 
jz and v has a memory 


Kuv = ela cos(~, — Pu) T Kus (26) 
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Vuv 


Fig. 14 Integrator circuit representing the Kuramoto model (left) and a memristive coupling ele- 
ment to describe the input signal of integrator u from integrator v (right) 


where € is the learning rate, œ determines the weighting of new information. Here, 
the cosine function seeks to model the associative learning rule, as it is maximizes 
when the phases p, and ¢,, are similar and minimized when the phase shift is [36]. 
For the circuit synthesis process, details are elaborated in [37]. The two circuits of 
Fig. 14 are synthesize the Kuramoto model as in (25). The underlying equation of 
the left circuit is 


ú, = ——@ — iy, (27) 


where To, Jo, Co are normalization constants with the units of a time, a current and 
a capacitance, respectively. A detailed analysis on a Kuramoto model with adaptive 
coupling is provided in [38]. It can already be seen that (27) synthesizes the first 
equation of (25) under the condition that 


1 m , a) 
iS Kuv sin | ——— ], (28) 


where Up is a normalization constant with the unit of a voltage and u, is associated 
with g,. Note that the negative sign has been incorporated in the argument of the 
sine function. 

The right circuit of Fig. 14 possesses the underlying equation 


Juv = uv (Zav Vuv)Upvs Vuv = Up — Uy, (29) 


with W,» = 1/M,,,. The goal of the memductive interconnection is to model the v-th 
element of the sum in (25). This is achieved when the coupling strength is associated 
with the state of the memductor and the phase difference with the voltage over the 
memductor, such that Kav = Zyy and Qu — Pv = Vav. For the memductance it follows 
that 


] v 1 . v 
Wee = gait Gs, (30) 
m Uo 
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Fig. 15 Setup of four synthesized and fully coupled Kuramoto oscillators with the memductances 
incorporating the Hebbian learning rule 


where Go is a normalization constant with the unit of a conductance. It must be 
mentioned however that (30) allows for W,» < 0 and hence is not passive in general. 
Such memristors are called active memristors [26]. The Hebbian learning rule (26) 
is then synthesized by the proper modeling of the memductor’s state equation 


: Vuv 
Zim =E E cos (w) — zu : (31) 


Figure 15 shows a setup synthesizing four Kuramoto oscillators with a fully coupled 
interconnection network [37]. 

In the following, a condensed description of the memductive interconnection 
network will be given before the wave digital model is established. To obtain a 
general description of an arbitrarily large number of connected Kuramoto oscilla- 
tors m, the voltages and currents are summarized by vectors u = [u,..., Um]? and 
i=[iy,...,in]’, respectively. Notice in Fig. 15 that the orientiation of the currents 
Juv is chosen such that the they originate at the lower level subsystem and are directed 
towards to the higher level subsystem. This allows for a representation of the inter- 
connection structure by the incidence matrix N. Kirchoff’s node and current laws 
are then given by 


i=Nj, u=N"v, (32) 
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where j is the vector of all currents j,,, and v is the vector of all voltages v,,, of 
the interconnections with u, v = 1,...,m, u < v. Utilizing this notation, (29) is 
rewritten as 


f= Wz, vv, WG, v) = diag (Wyo (Zav, Pv) (33) 


L,v=1, u<v 


With the help of (32), the relationship between the oscillator voltages and currents 
u, i and the memductors of the interconnection network is described by 


i=Wu, wih W=NW(z,v)N’, (34) 


where W = Wz, v) contains both the information about the memductances and the 
topology. With these insights it is possible to not only form a vector-based repre- 
sentation of the synthesized Kuramoto model, but to also obtain the corresponding 
wave digital model, cf. [37]. There, Rc = T/2Co1, Rw = Rw1 and consequently 
Rs = [R z: +R z The value of Rw can essentially be chosen arbitrarily, as long 
as Rw > 0, but it should have a value close to the middle of W„,’s range. Doing so 
typically results in a good numerical condition of the scattering matrix Sw, where 


i=Wu & aw= Swbw, with Sw = [R; + ŴR; - WI. (35) 


In the following, an application example of the synthesized Kuramoto model in 
the context of an optical illusion is presented. For this, the optical illusion depicted 
in Fig. 16 is considered. Here, the setup of Fig. 15 models the four relevant aspects 
of the optical illusion, where M4, M3 account for the water and cloud (background), 
respectively. M2 accounts for the bottom part of middle pillar of the object and M4 
accounts for both the top part of the middle pillar and the cloud background. It 
becomes apparent that the state of M4 and whether it is associated with the cloud 
or the middle pillar is the key factor of why the whole setup is an optical illusion. 
As the bottom part of the middle pillar and the ocean are spatially close, it appears 


Fig. 16 An optical illusion — pona 
being modeled by the setup ( y A 
of Fig. 15 = 
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Fig. 17 Four oscillators form two synchronized groups with the latter being in an anti-phase 
configuration, explaining the contradiction of the optical illusion 


natural to initialize the phases of the corresponding Kuramoto oscillators close as 
well. The same holds true for the archway and the cloud. Since the ocean and cloud 
are spatially far away from each other, their phase difference is comparatively big 
initially. Hence, g; = 0, g2 = —0.17,¢3 = x and g4 = 0.97 was chosen. Moreover, 
the learning rate was chosen € = 0.1, the weighting of new information œ = 0.1 and 
the intrinsic radian frequency œ = 1s~!. The according emulation results are shown 
in Fig. 17. It can be observed that due to the initial conditions, the oscillators M4 and 
Nh representing the ocean background and the lower part of the middle pillar fire 
together. And as expected, due to “cells that fire together, wire together” [32], these 
two oscillators synchronize. This is interpreted as being an association of the bottom 
part of the middle pillar with the ocean. The same association takes place with with 
the top part of the pillar and the cloud, represented by oscillators M3 and M4. This 
example might indicate that the optical illusion works due to the spatial correlation 
of the contradictory association between object and background. This contradiction 
is represented by a large phase shift between the two synchronized subgroups in 
Fig. 17. Of course, a human is able to detect the contradiction after a short period of 
time and to determine the origin of the optical illusion. 


6 Conclusion 


Due to their inherent massive parallelism in a signal processing sense, electrical cir- 
cuits in general are candidates to replicate the computing power as well as energy 
and spatial efficiency of biological systems. In order to do so, the underlying biolog- 
ical systems must be understood and then abstractly transferred to hardware circuits, 
making bio-inspired computing a highly interdisciplinary field. Before circuits can 
be manufactured, theoretical circuits must be designed to abstractly mimic biological 
behavior and serve as a proof of concept. This purpose has been fulfilled in a variety 
of application presented in this chapter. The wave digital concept has been used as an 
emulation technique to digitally replicate these theoretical circuits. Since memristors 
are the centerpieces of all these circuit concepts and currently hard to fabricate with 
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the current state of technology. Hence, software-based preinvestigations are useful 
to shorten development cycles, especially since the wave digital concept is real-time 
capable and offers in-operando parameter manipulation. For example, this has been 
exploited to optimized parameters of the well-known Morris-Lecar model in order 
to show a more biologically meaningful behavior. Further applications included a 
gait pattern generator, for which a unidirectional synapse has been developed. Also, 
two- and multi-neuron communications has been investigated on an abstract level, 
where the circuit concept for the latter also exhibited self-healing to repair synaptic 
damage and hence showed a rich behavior in the context of neuroplasticity. Lastly, 
an optical illusion was investigated to mimic an elementary decision problem. 

In summary, the field of bio-inspired computing has the potential to tackle prob- 
lems, which are too complex for current computing concepts. The memristor is a 
novel electrical circuit element that enables computations with and in memory and 
could be the key component to enable bio-inspired circuits. The bottlenecks, which 
need to be overcome to manufacture high performance circuits that mimic biological 
behavior is twofold: First, the understanding of the way biological systems function 
must be increased to identify its general and abstract mechanism, which then need to 
be modeled in an electrical circuit. Second, the manufacturing process of memristors 
must become more reliable and precise. 
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Nanoparticles Towards Complex pieci 
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Abdou Hassanien, Rainer Adelung, Franz Faupel, and Alexander Vahl 


Abstract Novel hardware concepts in the framework of neuromorphic engineering 
are intended to overcome fundamental limits of current computer technologies and 
to be capable of efficient mass data processing. To reach this, research into mate- 
rial systems which enable the implementation of memristive switching in electronic 
devices, as well as into analytical approaches helping to understand fundamental 
mechanisms and dynamics of memristive switching is inevitable. In this chapter, 
memristive switching based on Ag metal filament formation is discussed throughout 
different scales, providing insights on the stability of metal filaments and the onset 
of collective behaviour. An unconventional cAFM approach, which intends to inte- 
grate the memristive system directly on the apex of the cantilever instead of usual 
contacting is presented. This facilitates the nanoscale probing of filamentary memris- 
tive switching dynamics on long time scales for the purpose of basic research, 
which is demonstrated by an archetypical electrochemical metallization (ECM) based 
system consisting of Ag/SizN4/Au. Further, the application of AgAu and AgPt noble 
metal alloy nanoparticles (NPs) for memristive devices is discussed with special 
focus on the device scalability. For the smallest scale it is shown, that a single 
AgPt-NP encapsulated in SiO, operates via stable diffusive switching. Finally, two 
concepts for the self-assembled fabrication of NP-based memristive switch networks 
are evaluated regarding to collective switching dynamics: A sub-percolated CNT 
network decorated with AgAu-NPs and a Ag-NP network poised at the percolation 
threshold. The hybrid CNT/AgAu-NPs networks exhibit a mixed form of diffusive 
and bipolar switching, which is very interesting for tailoring the retention time, while 
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the networks dynamics of percolated Ag-NP networks are governed by ongoing tran- 
sitions between a multitude of metastable states, which makes them interesting for 
reservoir computing and other neuromorphic computation schemes. 


Keywords Memristive devices + Diffusive switching - Self-assembled networks - 
Nanoparticles + Neuromorphic engineering - Dynamic networks 


1 Introduction 


For several decades, the increasing demand for computational data processing was 
satisfied by miniaturization of the transistor, which plays the role of the funda- 
mental building unit in modern computer technology, through extreme manufac- 
turing techniques and engineering of materials. Alongside with current developments 
like Internet of Things or autonomous driving for which massive data processing is 
required, the demand for computational power presumably cannot be met by conven- 
tional computer technologies in the near future [1]. One reason for this relates to 
physical limits as the size of a single transistor approaches atomic scale like a funda- 
mental integration density limit or information lost via inadvertent tunneling currents 
[2]. Another reason is the strict separation of information processing and storage units 
predefined by the von-Neumann architecture of modern computer systems, since the 
duration for massive data transfer between both units evolved into a bottleneck. In 
foreseeable future, a gap in the computer technology must be filled in order to remain 
capable of dealing with an increasing massive amount of data. The field of neuromor- 
phic engineering comes along with unconventional computation schemes, which aim 
to overcome the aforementioned limits by implementation of biological computation 
principles in novel kind of hardware [3, 4]. Memristive devices play a fundamental 
role in the realization of bio-inspired hardware. They are two-terminal passive circuit 
elements, whose internal state (represented by the resistance of the device) is sensitive 
to external voltage or current stimuli [5]. This property allows the implementation of 
key features for neuromorphic computation like parallelism, in-memory computing 
and plasticity [2]. Moreover, the functionality of memristive devices can be expanded 
towards memsensing, which means a combination of memristive and sensitive prop- 
erties [6]. Tremendous work was dedicated to the development of material systems 
showing memristive behavior and understanding of nanoscale processes which are 
responsible for the memristive switching property, which resulted in a multitude of 
memristive material systems exhibiting diverse memristive switching dynamics [7]. 
One example is analog non-volatile switching, which is a potential electrical char- 
acteristic for long-term memory in neuromorphic circuits and enables the hardware 
implementation of artificial neural networks. Diffusive switching possesses a volatile 
threshold-like dynamic beneficial for short-term memory realization and incorpora- 
tion of non-linearity in neuromorphic circuits, which are both inherent for brain 
functionality [8, 9]. 
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Besides valence change [10] and phase change mechanisms [1 1], the electrochem- 
ical metallization (ECM) is of special interest to engineer materials with memristive 
properties. ECM relies on redox-reactions and ion transport of active metallic species, 
like Ag or Cu, in a nanoscale switching gap upon application of an external electrical 
field [12]. The switching dynamics in ECM devices is dominated by the reconfig- 
urable formation and rupture of a thin filament composed of the active metallic 
species. The operation of ECM devices can be constrained into a diffusive regime by 
utilization of a strict limitation of current flowing through the device during its oper- 
ation or by limiting the amount of the active metallic species [13]. The understanding 
and control of the complex processes governing the filament evolution during device 
operation is still a current challenge. Especially, the inherent stochasticity in the 
switching dynamics (in terms of variation of switching voltages from cycle to cycle) 
is merely insufficiently studied. The stochasticity may impede the development of 
certain applications like selector devices, but it may also be exploited like in the 
design of true random number generators [14, 15]. 

A critical issue in the development of neuromorphic hardware is the challenge 
of upscaling from individual memristive units to networked assemblies with an 
enormous degree of complexity. Promising approaches under circumvention of vast 
wiring procedures rely on self-assembly of memristive switching units. Reports have 
shown, that nanoparticles (NPs) of Au [16, 17] or Sn [18] or Ag-nanowires [19, 20] 
can be applied as fundamental memristive building units for self-assembled networks. 
Such networks are capable of responding via complex dynamical switching patterns 
featuring short-term memory and temporal correlations, which are both key require- 
ments for concepts like reservoir computing [21]. A feasible strategy to maximize 
the degree of complexity in self-assembled networks of switching units, and there- 
fore to approach neural complexity, is to make use of criticality, where the network 
dynamics are governed by complex correlated patterns, similar to those observed in 
neural tissues [22, 23]. Networks of memristive switching units can be operated at 
critical dynamics, when the connectivity inside the system is poised at the perco- 
lation transition [24]. Developments in the design of material systems and deeper 
characterization of emergent properties in networks of memristive units could be a 
significant advancement for bio-inspired computation strategies. 

In this chapter, we discuss basic studies on Ag-based filamentary switching and 
approaches for device implementation on fundamentally different scales, ranging 
from single switching junctions to complex network behavior. Firstly, an unconven- 
tional conductive atomic force microscopy (cAFM) approach giving deep insights 
into the long-term switching dynamics and inherent stochasticity of a single Ag- 
filament is presented. Moreover, the application of Ag-based bimetallic noble metal 
NPs as individual memristive switching units is discussed. Major emphasis is put 
on implications which arise, when the memristive system is scaled up from a single 
switching unit towards macroscopic assemblies of switching units, i.e. up to which 
scale does the characteristics of individual NPs retain and when does collective 
behavior emerge. Therefore, discussions on macroscopic assemblies realized by 
two different approaches in which Ag-based noble metal NPs are incorporated 
into complex networks are given in the last section: The first approach comprises a 


222 N. Carstens et al. 


randomly assembled CNT network with Ag-based NPs as localized switching units 
while the second approach involves self-assembled networks of Ag-NPs poised at 
the percolation threshold. 


2 Deep Insight into Single Filament Switching Via 
an Unconventional cAFM Approach 


Progress in the development of novel neuromorphic hardware is directly reliant on the 
understanding of the fundamental units causing the memristive action. In ECM-based 
devices, as well as in self-assembled memristive switch networks, this fundamental 
unit is a single metallic filament, which defines an either conductive (low resistive 
state, LRS) or insulating (high resistive state, HRS) state of a nanogap. In ECM- 
based devices, there is in most cases solely one single filament, which dominates 
the behavior of the whole device [25]. Considering the typical diameter of an active 
filament in the order of 10 nm, this means that all memristive action during device 
operation takes place highly localized on the nanometer scale and is independent 
of the devices’ geometrical dimensions defined through the fabrication [26]. The 
analysis of inherent stochastic features in the switching dynamics of single filaments 
are of special interest, because they set fundamental constraints on possible appli- 
cations. On the contrary, in self-assembled memristive switch networks, filamentary 
switching is spatially distributed over a huge number of nanogaps throughout the 
network and the network response results from a complex interplay between the states 
of different filaments. In such networks, studies on single filamentary switching are 
likewise inevitable to understand the role of single elements in the collective network 
dynamics. 

CAFM raised to an essential tool probing electrical properties on the nanoscale and 
was commonly used to make localized studies on switching filaments [27]. However, 
a prominent drawback of the conventional cAFM method is associated to the thermal 
drift of the cantilever, because it imposes uncertainties regarding the position of the 
cantilever relative to the filament under investigation. The extent of the thermal 
drift is usually more than 10 nm h7!, which exceeds the typical dimensions the 
filament. This is especially problematic when the long-term electrical response of a 
single filament shall be measured. To circumvent this issue, an unconventional cAFM 
approach to probe the long-term response of the memristive switching of a single 
filament can be applied [28]. Instead of conventional contacting, the memristive 
device is integrated directly on the apex of a conductive cantilever. In this way, 
the functionalized cantilever can be brought in reliable contact using the cAFM 
instrumentation to any surface which acts as an inert counter electrode. This approach 
is sketched in Fig. 1. 

Archetypical memristive systems consisting of a few nm thin Ag layer covered by 
a dielectric layer on Si3N4 were deposited on the apex of cantilevers by conventional 
physical vapor deposition (PVD) techniques. The basic coating of the cantilever 
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Fig. 1 Unconventional cAFM approach in the present study. Instead of conventional contacting, 
the memristive system is integrated on the apex of the cantilever via PVD methods. The func- 
tionalized cantilever is brought into contact with inert counter electrode surfaces (Au and ITO in 
this work) resulting in an ECM-based memristor configuration. As the filamentary switching is 
directly constrained to the apex, this approach allows more reliable long-term measurements on 
the nanoscale via conventional cAFM instrumentation. Reproduced under CC-BY 4.0 license from 


[28] 


was chosen as Au to guarantee a good conductivity and electrochemical passivity of 
the cantilever. The configuration of functionalized cantilever contacted to the inert 
counter electrode surface defines a complete ECM-device to be probed. Cantilevers 
functionalized with Ag/Si;N, were investigated under two different configurations: 
Having either Au or indium tin oxide (ITO) as inert counter electrode. This approach 
offers two decisive advantages for the purpose of basic research. Firstly, the memris- 
tive action through the filament is constrained directly to the apex of the cantilever. 
Consequently, thermal drift causing lateral motion of the cantilever over the counter 
electrode surface does not lead to any separation of probe and switching locality. As 
thermal drift effects are mitigated, reliable nanoscale long-term measurements of the 
filamentary switching activity are enabled using conventional cAFM instrumentation. 
Another advantage of this approach concerns the flexibility in choosing the experi- 
mental configurations, because an identical functionalized cantilever can be brought 
into contact to diverse counter electrodes. This enables strategies to investigate the 
switching activity of the very same filament on different interfaces. 

In the following, the long-term switching of a single filament is discussed based 
on data gathered from a cantilever functionalized with Ag/Si3N,4 contacted to a 
Au surface. The contact was held at a well-controlled force of 1.2 nN to provide 
non-invasive measurement conditions. The counter electrode was biased via voltage 
sweeps between —2.6 and 2.6 V and current responses were recorded continuously 
over 18 h resulting in over 12,000 switching cycles. During the measurement a 
1 GQ serial resistance was applied to limit the current and protect the cantilever 
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from Ag over-diffusion. The filamentary switching mechanism taking place sharply 
localized at the apex and an exemplary switching cycle are shown in Fig. 2a and 
2b, respectively. Initially, under zero-bias conditions, the sample is in its HRS since 
no stable filament can exist between probe and counter electrode. Upon biasing 
the counter electrode (towards either polarity), electrochemical oxidation of Ag is 
triggered at anodic sites in the sample, leading to a release of Ag* -ions into the 
dielectric layer. The Ag* -ions are mobile and can migrate in the SiN, layer until 
they become reduced at cathodic sites in the sample where they aggregate as metallic 
silver. Through this mechanism, metallic Ag is reconfigured in the dielectric layer 
until a continuous filament bridges the SiN; layer, which causes a switching event 
to the LRS. This moment is denoted as a SET event in the current response and 
the current level (highlighted in blue) is solely limited by the external 1 GQ serial 
resistance. When the voltage bias is reduced coming from the LRS, a RESET event 
is observed before reaching zero-bias. The physical mechanism behind this is the 
spontaneous disintegration of the Ag filament into individual clusters caused by 
interface energy minimization [13]. The fact that the filament is only stable above 
a certain threshold voltage is the most prominent feature of diffusive memristive 
switching. Among all cycles in the long-term measurement, 95% of them showed 
clear diffusive switching, which indicates the stable and non-invasive measurement 
conditions of this approach. It should be noted, that the incorporated 1 GQ resistance 
is the reason why the switching is constrained into the diffusive regime. On the one 
hand side, the limited current only allows for the formation of thin filaments which 
are more prone to spontaneous disintegration. On the other hand, in the moment of 
SET switching the major part of the externally applied electrical potential shifts to 
the serial resistor, which significantly reduces the field across the filament. 

In another experiment, the consequences of having Au or ITO as counter elec- 
trode interface are investigated. For this purpose, the Ag/Si3;N4 system of an iden- 
tical cantilever was brought into contact firstly with Au and subsequently with ITO. 
The decisive advantage of this approach is that an identical active filament was 
probed in consecutive measurements under different ECM-based device configura- 
tions. Therefore, all influence coming with the active electrode or dielectric layer 
(such as defect structure or filament morphology), which might disturb the compa- 
rability of different configuration, are ultimately kept constant. In contrast to that, 
in conventional cross-point devices, an investigation of the variation of different 
counter electrodes is only possible at the expense of losing the direct comparability 
in terms of the exact same filament [29, 30]. Figure 2c depicts the current response 
during operation on Au (black) and ITO (red) over 50 consecutive cycles for each 
electrode. It can be seen that interfacing to Au drives the switching window to lower 
voltages. A representative switching cycle (scan speed: 4 s per cycle) drawn in bold 
line shows a switching window between a RESET voltage of 0.3 V and a SET voltage 
of 1.9 V in the positive regime. Further, it was detected that upon interfacing to Au 
the LRS follows a linear characteristic, which reveals that contact between filament 
and counter electrode surface is purely ohmic. Switching the contact to ITO in the 
subsequent measurement resulted in fundamentally different electrical characteris- 
tics. In this configuration, the switching window is shifted to higher voltages (above 
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Fig. 2 a Physical mechanisms taking place during filamentary switching in the sample. b Repre- 
sentative cycle from the long-term measurement. The operation of the Ag/Si3N4 system integrated 
on a cantilever and contacted to a Au surface resulted in diffusive switching characteristics due to 
the 1 GQ serial resistance. The distinctive feature of diffusive switching is, that the LRS is only 
stable above a certain threshold voltage and volatile at zero-bias conditions. c Operation of an iden- 
tical filament from one functionalized cantilever on different counter electrode surfaces: Au (black) 
and ITO (red). A non-linearity in the current response was detected for ITO, which indicates the 
existence of an energetic barrier which was not observed during operation of the very same filament 
on Au. d Representation of 100 consecutive cycles among the long-term measurement. It can be 
seen, that SET and RESET threshold voltages have a statistical nature. Reproduced under CC-BY 
4.0 license from [28] 


2.5 V in the positive regime) and the current—voltage dependency exhibited a strong 
non-linearity. This indicates the occurrence of an energetic barrier at the interface 
between filament and ITO surface. 

The long-term measurements of an identical functionalized cantilever open up 
the possibility to study the inherent statistical nature of diffusive switching. For this 
purpose, a long-term characterization on Au was conducted. Figure 2d depicts 100 
consecutive cycles among the long-term measurement. It becomes apparent, that 
SET and RESET events are not triggered by fixed threshold voltages, but underlie a 
certain degree of stochasticity. Although great effort was invested to engineer fila- 
mentary systems with minimized switching variability, it is difficult to suppress it 
completely. Therefore, a deeper understanding of the statistical nature is inevitable. In 
Fig. 3a, all four threshold voltages per cycle (SET and RESET at either polarities) are 
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plotted against cycle number. A notable observation is, that the evolution of threshold 
voltages shows a significant trend ranging over hundreds of cycles in some regimes 
(highlighted as high correlation in the plot). However, in other regimes (highlighted 
as low correlation), the threshold voltages are subject to strong randomness. This 
observation suggests that there is partially a correlation in the switching characteris- 
tics in consecutive cycles and that the degree of correlation varies over the long-term 
measurement. To elucidate this, the linear correlation in the switching statistics was 
quantified by means of the Pearson coefficient. For both indicated regimes, the indi- 
vidual threshold voltages were related to the respective former value from the cycle 
before. This quantification is visualized in Figs. 3b and 3d for SET and RESET 
voltages with respect to positive polarity in the high correlation regime, respectively, 
and in Figs. 3c and 3e for the same characteristic voltages but in the low correla- 
tion regime. The values for the Pearson coefficient amount to 0.879 and 0.889 for 
SET and RESET in the high correlation regime, respectively, whereas they are 0.435 
and 0.581 for SET and RESET in the low correlation regime, respectively. This 
supports the observation, that the threshold voltages are not statistically independent 
on each other, but are subject to correlations in the switching dynamics, which puts 
fundamental constraints in the design of real devices. 


3 Noble Metal Alloy Nanoparticles for Diffusive Switching 


An interesting approach to confine memristive action to the nanoscale lies in the 
transition from conventional bulk planar active electrodes towards nanostructured 
active electrodes or NPs. Using nanostructured electrodes (e.g. nanocones) showed 
a beneficial effect on the switching uniformity in Cu-based filamentary memristive 
devices [31]. NPs have recently attracted attention as building blocks for memristive 
switching devices and are readily employed, either to enhance the reproducibility 
by tailoring the electrical field inside the dielectric layer [32] or to act as a source 
of mobile metallic cations [8, 33]. Using AgAu and AgPt alloy NPs, the benefits of 
predefined electrical field enhancement and restricted mobile cation reservoirs were 
combined, and diffusive memristive action was studied at different length scales, 
ranging from individual NPs to multistack nanocomposites. Restriction of the mobile 
species reservoir is crucial to achieve stable diffusive switching as the formation of 
non-volatile filaments is inseparably linked to larger mass transport of the mobile 
species. The use of bimetallic NPs is particularly promising, because the amount 
of mobile species at the switching location can be effectively limited through NP 
composition and size [33]. Figure 4 gives an overview of different device setups 
which were studied in this context. AgPt and AgAu-NPs fabricated from a Haberland- 
type gas aggregation source serve as fundamental memristive building units. To 
obtain alloy NPs such as AgAu or AgPt, the concept of gas phase synthesis in a 
magnetron-based Haberland-type gas aggregation source [34—36] was expanded by 
the introduction of the multicomponent target approach. Here, instead of a planar 
bulk target, a custom-made target is applied, which consists of a Ag target with a 
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Fig. 3 Switching statistics extracted from the long-term measurement of a Ag/Si3N4 system on Au. 
a Characteristic threshold voltages (SET (red) and RESET (green) at positive polarity, SET (blue) 
and RESET (black) at negative polarity) plotted versus cycle number. Two regimes are highlighted: 
One showing systematic trends in the evolution of threshold voltages (labeled as high correlation) 
and one showing strong randomness (low correlation). The linear correlation was quantified by the 
Pearson coefficient in both regimes by relating the individual threshold voltages to the respective 
former value from the cycle before for SET and RESET in the high correlation regime (b, d) and 
SET and RESET in the low correlation regime (c, e). Reproduced under CC-BY 4.0 license from 
[28] 
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Fig. 4 Device setups under investigation. AgAu and AgPt-NPs act as fundamental memristive 
building units and are encapsulated into SiO2 for a complete ECM-based memnristive device. Left: 
Experimental setup to test the switching performance of individual NPs via cAFM. Right: The 
upscaling capabilities of the NP-based approach was tested on multiple stacks of NPs and SiO2 
layers via a macroscopic probe. Reproduced under CC-BY 4.0 license from [33] 


trench and embedded Au or Pt wires. Using this methodology, the material usage of 
the more precious metal was significantly enhanced and the NP alloy composition 
can be tailored and controlled in-operando. More details regarding the NP deposition 
methodology can be found elsewhere [35, 36]. Encapsulation of the alloy NPs in a 
dielectric layer consisting of SiOz (deposited by reactive pulsed DC sputtering from 
a Si target in the presence of an Ar/O, atmosphere) completes the memristive device 
setup. 

In contrast to the vast majority of ECM-based devices, the reservoir of active 
species (Ag in this case) has not the form of a continuous electrode, but is embedded 
into a dielectric layer between two inert electrodes. A major focus of this study was 
to explore the scalability of this concept which is one key aspect concerning device 
implementation. Therefore, this concept was tested on two different scales: Firstly, 
the switching performance of an individual alloy NP was tested via cAFM, which 
embodies the lowest limit of the scalability range. Secondly, macroscopic devices 
consisting of multiple stacks of NPs and SiO, layers were fabricated to characterize 
the upscaling capabilities. 

The NP-based approach used to implement memristive devices on the nanoscale 
offers several advantages. The location of NPs inside the dielectric layer efficiently 
predefines the path where the filament is formed, which is essential for the device 
integration on the lower nanoscale. The strong localization of memristive action is 
additionally enhanced by the inherent electrical field concentration at the surface of a 
NP. Since the mechanisms leading to filament formation are driven by electrical fields, 
a substantial facilitation of memristive switching by using NPs is expected. Further, 
application of alloy NPs instead of the elemental counterparts provides additional 
degrees of freedom to tune the memristive action. For the alloy system presented 
here, AgAu and AgPt, Ag is the active species, whereas to more noble component, 
Au or Pt, is completely inert and will remain at its location inside the dielectric layer, 
which is expected to facilitate the long-term anchoring of the filament path. 

To understand the memristive action of alloy NPs as building block for memristive 
devices, first the switching at the level on an individual NP has to be understood. 
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Fig. 5 Essential results from the memristive performance characterization of a single AgPt-NP 
encapsulated in SiO? via voltage sweeps between —7 and +7 V probed by cAFM. a shows a plot of 
70 consecutive cycles which are representative for the diffusive switching behavior of a single NP. 
b shows the threshold voltage statistics for over 2000 consecutive cycles. Adapted under CC-BY 
4.0 license from [33] 


For this purpose, a stacked system consisting AgPt-NPs encapsulated in SiO. was 
prepared through consecutive deposition steps of 8 nm SiO», AgPt-NPs at sparse 
filling factor and 2 nm SiO2. A cAFM setup was applied to identify the position of 
a NP and to measure the current response at the location of an individual AgPt-NP 
via driving voltage sweeps between —7 and +7 V through the conductive tip, while 
the common Au thin film served as back electrode. A serial resistance of 101 MQ 
was applied to this measurement to limit the current flowing through the system. In 
this measurement, stable diffusive memristive switching was observed for over 2000 
cycles. Essential results of the measurement are depicted in Fig. 5. 

The memristive behavior of the single AgPt-NP matches diffusive switching and 
consequently follows the identical qualitative characteristics as described for single 
filament switching in the previous section. This is represented in Fig. 5a, where 
70 consecutive cycles among the measurement are plotted. The threshold voltage 
statistics for over 2000 consecutive cycles are given in Fig. 5b. Also in the case of 
NPs, the switching characteristics underlie a considerable degree of stochasticity. 
Interestingly, no initial electroforming step had to be performed to achieve a stable 
switching regime. The origin of the diffusive switching behavior in the single AgPt- 
NP system is not merely determined by the external current limitation (like in the 
functionalized cantilever devices in the previous section), but rather by the absence 
of bulk Ag electrodes. The alloy NP approach allows for a severe limitation of the Ag 
amount (i.e. by small NP diameter or low ratio between Ag and noble alloy compo- 
nent) and therefore provides degrees of freedom to constrain the system into diffusive 
switching regime by supplying not enough Ag to build a non-volatile filament. 

Further, the scalability of the NP-based approach was tested through studies on 
macroscopic devices. For this purpose, samples consisting of multiple stacks of SiO2 
layers and either AgAu or AgPt-NPs were fabricated and characterized in analogy 
to the cAFM measurements. To account for effects arising from the thickness of 
the SiO» separation layer, an effective SiO% thickness of 2 nm between the NP 
depositions was chosen for the AgAu sample whereas it was chosen as 4 nm for the 
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AgPt sample. A soft Pt probe having a diameter of 125 um was applied to contact the 
samples. The samples were stressed again via voltage sweeps under current limitation 
with a 1 MQ serial resistor. Figure 6a and 6b exhibit the current responses and 
threshold switching statistics measured for the macroscopic devices based on AgAu 
or AgPt-NPs, respectively. Generally, it can be observed, that the diffusive switching 
characteristics as known from a single NP is preserved, although multiple switching 
junctions inside the sample are expected to contribute to the filament formation. 
Comparison of the different alloy systems show, that the HRS of the AgAu sample 
allows a measurable conduction whereas the HRS of the AgPt sample is hidden 
in the noise level of the experimental instrumentation. These observations could 
be connected to the different thicknesses of the SiO» separation layer. In the AgAu 
sample, the lower SiO thickness possibly allows significant leakage currents whereas 
the 4 nm SiO, layer thickness in the AgPt sample results in effective insulation. 
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Fig. 6 Upscaling of the NP-based approach. Macroscopic devices were fabricated by multiple 
stacking of SiOz layers and AgPt or AgAu-NPs. The samples were contacted by a macroscopic Pt 
probe (diameter of 125 um) and characterized via voltage sweeps. a and b show switching cycles at 
positive bias and extracted switching statistics for systems with AgAu and AgPt-NPs, respectively. 
Reproduced under CC-BY 4.0 license from [33] 


Memristive Switching: From Individual Nanoparticles Towards ... 231 


4 Distributed Ag-Based NP Switching in Memristive 
Networks 


Alongside with the profound characterization and device engineering of single 
memristors, further sophisticated challenges arise from the question how to build 
functional networks with a magnitude of memristive switching units. These chal- 
lenges can be tackled by two different paradigms: Either by a regular and precisely 
engineered arrangement of hardwired memristive devices, like it is done in cross-bar 
arrays, or by reliance on self-assembly mechanisms to achieve a random dispersion 
of memristive switching units throughout a network. In the last section, two strategies 
for the fabrication of Ag-NP based memristive switch networks under self-assembly 
principles will be discussed. 


4.1 Sparse CNT Networks with Implanted AgAu 
Nanoparticles 


From investigating single NPs or vertical stacks of only a few NPs, in this section, 
the transition of NP-based memristive switching to the horizontal orientation is 
discussed. For achieving the nanoscaled distances required for memristive switching 
phenomena vertically, a variety of thin film deposition techniques are already well 
established. However, in a lateral orientation only sophisticated techniques like e- 
beam lithography [37, 38] offer the necessary resolution to produce nanoscaled 
gaps between electrodes, that allow for memristive switching. In this section, a new 
approach for the fabrication of lateral nanogaps as well as the memristive switching, 
when combined with bimetallic NPs described in the previous section, is discussed 
[39]. 

This approach is based on depositing carbon nanotubes (CNTs) between elec- 
trodes fabricated by conventional UV-lithography where the spacing between the 
electrodes is of several micrometers. The deposition is controlled such, that the CNTs 
form a sparse network and a subsequent Joule heating step removes any existing 
conductive pathways between the electrodes. The network then exhibits nanoscaled 
gaps between CNTs allowing for memristive switching phenomena. To fulfill its 
expected functionality, the CNT network has to meet certain requirements: 


e The network must fill the space between the electrodes, so that all electrodes are 
in electrical contact with the CNT network. 

e The CNTs must be finely dispersed, so that there are no dense agglomerations of 
CNTs, as those would impede the subsequent Joule heating step. 

e The network must be below the percolation point but dense enough, so that the 
distance between individual tubes is in the nanometer range. 
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e The CNTs must not be heavily coated by surfactants or other additives, which 
would hinder removing short-circuiting paths during the Joule heating step. 


Based on these requirements, a custom-made CNT dispersion has been developed 
by mixing ethanol with dry, pristine CNTs and a small amount of PEDOT:PSS. The 
mixture has been ultra-sonicated with a high-power sonicator to break up the CNT 
bundles and disperse them, whereas the PEDOT:PSS prevented a re-agglomeration 
of the CNTs and kept them finely dispersed. Despite the PEDOT:PSS acting as a 
surfactant, the low amount used is not detrimental to the Joule heating step. Using 
ethanol as a solvent allowed for a quick spin coating deposition in which single drops 
of dispersion are dropped subsequently onto the substrate under constant rotation, 
making the density of CNTs in the network well controllable. A sparse network of 
CNTs is shown in Fig. 7a. Conductive pathways in the CNT network, which have 
been short-circuiting the electrodes, were removed via Joule heating by applying 
voltages of up to 30 V. 

AgAu NPs have been deposited onto the CNT networks with a Haberland-type gas 
aggregation source, which has been discussed in the previous section. The percola- 
tion point of the NPs deposition process has been determined by in-operando current 
measurements. It has been used to adjust the NP density to be just below the percola- 
tion point, so that the distances between particles are in the lower nanometer range, 
which is indicated in Fig. 7b. CNT networks with implanted AgAu NPs showed 
ECM-type memristive switching with a switching behavior exhibiting a hybrid of 
diffusive and bipolar characteristics. Current—-voltage measurements are shown in 
Fig. 7c and d. Figure 7c depicts the diffusive switching mode, where the LRS is 
reset to the HRS when falling below a voltage threshold (cf. Figure 5). Additionally, 
Fig. 7d shows the capabilities for resetting the resistance state when cycled quickly 
into the reverse voltage regime, which is a feature of bipolar memristors. 

A time resolved current measurement is shown in Fig. 7e, which indicates the 
retention time i.e. the time until the HRS is reached after reducing the applied voltage. 
Below the voltage threshold, diffusion and surface tension lead to a collapse of the 
silver filament. However, the thickness of the filament, and thus the amount of silver 
in it, defines its lifetime [13]. Therefore, it is proposed that the deposited NPs provide 
a limited silver reservoir for filament formation. The amount of silver atoms is limited 
such that no stable filament can be formed. But there is still sufficient silver to form 
filaments that are thick enough to show a substantial lifetime. 

The retention time can also be described as the “memory span” of the device, 
as it remembers its resistance state. While in non-volatile memory applications, the 
retention time is supposed to be as high as possible to prevent data loss, a second- 
scale retention can be used for a short-term memory effect. Short-term memories are 
an efficient way of storing information that is only required for a limited amount of 
time, as it automatically forgets that information by itself again and thus does not 
have to be explicitly reset. 
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Fig. 7 Morphology and switching behavior of CNT networks with implanted AgAu NPs. a + 
b SEM micrographs of sparse CNT networks without a and with b deposited AgAu NPs. c Current- 
voltage cycles of a CNT network with NPs showing diffusive memristive switching behavior. d A 
current—voltage cycle showing the bipolar reset mechanism by applying reverse voltages (see arrow 
4). e Time-resolved current measurement showing the retention of the LRS. Adapted under CC-BY 
4.0 license from [39] 
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4.2 Nanoparticle Networks 


In addition to the networks of the previous section, another approach for large-scale 
memristive switch network implementation, which solely makes use of metallic NPs 
for building the network, is discussed in this section. Elemental Ag-NPs synthesized 
from Haberland-type gas aggregation source, which are similar to the memristive 
bimetallic Ag-based NPs as discussed in Sect. 3, are applied as fundamental memris- 
tive building units in this type of network. Network formation is done by random 
assembly (realized by the statistical nature of the deposition process) of the Ag-NPs 
between chemically inert electrodes. The resulting network consists of a complex 
arrangement of clusters of electrically connected Ag-NPs and potential switching 
gaps between different clusters. The system scale is defined by the electrode sepa- 
ration and can be easily set to large-scale systems in horizontal electrode geometry. 
A NP network system with maximized complexity (in terms of maximized amount 
of metastable network configurations) can be realized through careful tuning of the 
NP filling factor at the percolation transition [24]. A schematic setup of a percolated 
NP network is shown in Fig. 8a. In the subcritical regime, no network response is 
possible, because of missing current paths having the scale of the overall system. In 
addition to that, in the supercritical regime, the network complexity is expected to 
be strongly reduced due to the existence of stable current paths that dominate the 
network response. However, it should be noted that NP networks in the supercritical 
regime potentially are still able to show a dynamic response which results from the re- 
configuration of interface defects (such as grain boundaries) between connected NPs, 
as it is reported for Au-NP networks above the percolation threshold [17]. Exactly 
at the percolation threshold, which marks the critical regime of a phase transition, 
the network configuration is located at some point between none and full connec- 
tivity and allows for ongoing transitions between a multitude of different metastable 
network configurations [40]. Figure 8b shows schematically a Ag-NP network under 
indication of potential switching gaps. The state of the network is accessed by the 
conductance between both electrodes, which is determined by the configuration of 
all switching gaps across the network. Upon an external disturbance, e.g. application 
of voltage stimuli across the electrodes, localized memristive switching at individual 
switching gaps may be triggered, which lead to a redistribution of potential differ- 
ences across other switching gaps. As a consequence, memristive switching at a 
certain location could induce switching at other gaps in the network, resulting in a 
complex interplay of switching gaps distributed over the whole network. Figure 8c 
depicts the network response of a percolated Ag-NP network to a DC bias of 5 V. It 
can be seen, that the network dynamics are governed by ongoing transitions between 
a multitude of metastable conductance states over more than 1 h of operation. There 
is no indication that the network converges to a definable conductance state, which is 
in contrast to the lower-scale vertical NP-based devices in Sect. 3. Figure 8d shows 
the probability density of the observed conductance states for the vertical AgPt-NP 
devices from Sect. 3 (cf. Figure 6, right) and the percolated Ag-NP network. This 
comparison stresses out, that for the small-scale NP-based memristive devices well 
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differentiable conductance states (i.e. LRS and HRS) dominate the operation, which 
is not the case for the large-scale NP network. Instead, for the percolated NP network, 
the well differentiable conductance states are fading, which can be seen as an indi- 
cator for collective behavior of a multitude of switching junctions. Such a behavior 
is potentially interesting for neuromorphic computational concepts like reservoir 
computing, which require complex networks of interacting non-linear elements able 
to map input information into temporal dynamics in the network [41]. 


Fig. 8 Behavior of networks ( (b) 
comprised of Ag-NPs and 
poised at the percolation 
threshold. a A schematic 
percolation curve showing 
classification of the network 
states into sub- and 
supercritical and critical 
regime depending on the 
filling factor. b is showing 
schematically a critical 
network state, where no 
global connectivity persists, 2.0 
but the network state is 
defined by the configuration 
of a multitude of switching 
gaps. c exhibits the network 
response upon application of 
a constant external bias of 

5 V. d shows the distribution 
of distinct resistive states for 1.2 

a AgPt-multistack device as 

discussed in Sect. 3 (left) and 0 20 40 60 
extracted from the Ag-NP Time [min] 

network response shown in ( 8 

(c) (right) 


critical 


oO 
[e] 
= 
T 
U 


Conductance [a.u.] & 


C Switching Gap 


Time [a.u.] 


~ 
A 
~ 


1.8 


Current [mA] 
i 
[e2] 


Probability Density & 


8 10 12 2.5 3.0 
log(Resistance [Q]) Resistance [kQ] 


236 N. Carstens et al. 


5 Conclusion 


In this chapter, Ag-mediated reconfiguration of resistance states have been observed 
throughout different device implementations and system scales, ranging from indi- 
vidual filaments to complex assemblies of switching gaps. Fundamental research 
into the long-term switching dynamics of an individual Ag-filament via cAFM can 
be considerably facilitated through direct implementation of the memristive material 
system at the cantilever apex instead of conventional contacting, because it mitigates 
loss in data significance by thermal drift effects. This approach was exploited to 
study the cycle-to-cycle variability of filamentary diffusive switching dynamics. It 
was observed that the variability in the switching dynamics of an individual filament 
cannot be seen as purely stochastic, but potential correlation effects must be taken 
into account. Further, AgAu and AgPt alloy NPs have been found to be promising 
and versatile building blocks for diffusive memristive devices with a broad range 
of switching properties. Investigations via cAFM on individual Ag-based alloy NPs 
have shown, that already a single NP can act as a fundamental memristive building 
unit. Application of NPs for memristive devices opens up several design opportuni- 
ties, such as inherent field enhancement at the surface of the NP, high localization 
of the memristive action at the location of the NP in contrast to bulk electrodes and 
precise regulation of the number of active species through NP size and composition, 
which is important to stabilize the diffusive switching regime of the device. Finally, 
two approaches to build large-scale memristive switch networks by self-assembly 
principles with a different degree of sparseness are discussed. The first approach is 
comprised of a sparse CNT network, which establishes a static network topology, 
and AgAu-NPs, which enable memristive switching between adjacent CNT sites in 
the network. Such hybrid CNT/AgAu-NPs networks were shown to exhibit a mixed 
form of diffusive and bipolar switching. The second approach treats networks only 
comprised of Ag-NP with a filling factor around the percolation threshold. The Ag- 
NP networks differ from their lower-scale counterparts regarding the non-existence 
of stable and definable conductance states. Instead of a clear differentiation into 
LRS or HRS, the networks dynamics of percolated Ag-NP networks are governed 
by ongoing transitions between a multitude of metastable states, which makes them 
interesting for neuromorphic computation schemes, where spatio-temporal mapping 
of information into complex patterns is required. 
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Abstract In neural systems, plasticity can be found throughout a variety of scales, 
ranging from local synaptic plasticity between two neurons towards long-range 
connections and global plasticity within larger neuron assemblies. While memristive 
devices have attracted a lot of attention as a potential neuromorphic analog to repre- 
sent local synapses and are regarded as promising building blocks for neuromorphic 
engineering, long-range connections and globally mediated aspects like homeoplas- 
ticity are not yet widely considered for neuromorphic systems. In this chapter, photo- 
catalytic deposition is discussed as an approach to form metallic structures from a 
global liquid reservoir. In this context, the photocatalytic properties of TiO thin films 
are employed to reduce metallic species from the surrounding solution. This chapter 
will elucidate the fundamental process of photocatalytic deposition with photocat- 
alytic TiO; thin films and will showcase the applicability towards the formation of 
metallic structures at the example of arrangements of locally grown metallic Au 
structures. 


Keywords Photocatalysis - Self-assembly - Thin films - Strip conductor - Optical 
characterization 


1 Introduction 


Highly parallel systems such as neuronal networks combine processing and memory 
at the local synaptic level and are capable of completing complex tasks like pattern 
separation and completion with outstanding efficiency [1]. Neuromorphic engi- 
neering takes inspiration from neural networks, pursuing the aim of developing 
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novel, neuron-inspired, efficient computing approaches. Memristive devices, due to 
their capability of unifying processing and data storage, have been studied broadly 
in the context of the development of novel computing paradigms and in the field of 
neuromorphic engineering [2]. Commonly, memristive devices are applied as an elec- 
trical representation of synapses and arranged in crossbar arrays, which enable inter- 
esting applications such as in-memory computing [3]. In the field of neuromorphic 
engineering however, new paradigms are necessary to capture the entirety of signal 
processing in neural networks [1, 4]. One important aspect in neural networks is their 
capability of dynamic on-demand reconfiguration of synaptic connections between 
the individual neurons. This is impressively shown by the onset of synaptic pruning, 
which describes the decrease of the number of connections between neurons in a 
fully developed brain. Accordingly, the development of biological neural networks 
happens on two different time scales: The fast synaptic plasticity (at the level of local 
synaptic connections between two neurons) and the slow blooming and pruning (at 
the global level throughout the neural network). While the first aspect, the synaptic 
plasticity, has attracted considerable research interest and memristive devices have 
been readily applied to mimic a variety of synaptic properties, the latter aspect of 
global plasticity is still under research and novel approaches are necessary to incor- 
porate it into future bio-inspired hardware [4—6]. First approaches to replicate the 
global interactions of neuron assemblies are considering global connectivity through 
electrolyte gating, thus in a liquid medium [7]. In this chapter, metal line formation 
on a templated substrate from a liquid phase is discussed as an interesting approach 
with the potential to pave the way to achieve long-range, global plasticity. 

The formation of a metallic lines on a template substrate requires a method 
that allows the formation of well-defined, localized metal structures. An effective 
approach to creating lines with high accuracy is the fabrication by localized reaction 
on a pre-structured substrate. A suitable compound is needed that can react in an 
aqueous medium to cause deposition of the metal atom in such a manner to have 
a localized growth of the metal on the surface of a template. TiO, is a metal oxide 
which is known for its photoreduction ability and high photocatalytic property. In 
the presence of UV light, photoreduction of an electro positive metal ion will occur 
when TiO; is present in the medium. For example, Dawson et al. showed that HAuCl4 
can be reduced on surface of TiO? nanoparticles to form gold capped nanocompos- 
ites with TiO» core [8]. It was discussed that the concentration of the TiO2 core 
influenced the nanocomposite size and stability. Having a thin film of pre-structured 
TiO» on a surface could serve as a base for gold nanoparticle growth when placed in 
HAuCl, aqueous medium. This process is based on photocatalytic reduction of the 
gold precursor ions by TiO2 which would then result in the immobilization of the Au 
nanoparticles on areas patterned with TiO2 forming a structured metallic growth on 
the substrate. Thus, understanding the photocatalytic process involved is thus vital. 

Semiconductor-based photocatalysis covers a wide range of cutting-edge appli- 
cations in some important areas such as energy, environment, hygiene and disinfec- 
tion [9, 10]. Principally, photocatalysis is described as the integration of photoex- 
citation and surface catalysis. The photoexcitation contains light absorption and 
charge output, while surface catalysis concerns the utilization of photoinduced charge 
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carriers (electron (e~) and hole (h*)) for reduction or oxidation reactions [11]. 
This process is highly dependent on various parameters, including the photocat- 
alytic activity of semiconductors, light wavelength, and intensity as well as reaction 
temperature [12]. Recently, semiconductor-based photocatalysts (TiO2, ZnO, WO3, 
etc.) have gained considerable attention due to their high potential for solving envi- 
ronmental and energy problems [13]. However, these photocatalytic materials can 
be also considered to easily synthesize metallic or metal oxide structures (forming 
hybrid structures) for different kinds of applications (hydrophobic and hydrophilic 
coatings, sensor devices, reconfigurable connections, etc.) rather than only using 
them for environmental and energy applications [14]. 

In the recent decade, titanium (IV) oxide (TiOz2) is one of the most widely studied 
semiconductor photocatalysts due to its significant properties such as its strong photo- 
oxidizing power, non-toxicity, very good stability over a wide range of pH, chemical 
inertness, low-cost preparation as well as photocatalytic properties resulting from its 
bandgap energy and positions of the conduction band (CB) and valence band (VB) 
[15-17]. An overall mechanism for the photocatalytic reaction on TiO, surface is 
categorized as two main processes such as (i) the reduction of oxygen molecules 
(O2) and (ii) oxidation of water molecules (H2O) by photoinduced charge carriers 
(e7 and h*) [18]. Initially, the photocatalytic reaction is started when a photoelectron 
is excited from the filled VB of TiO2 to the empty CB as a result of irradiation. 
The absorbed photon must have the energy either equal or greater than the bandgap 
of TiO; (3.0-3.2 eV) to excite the electron in the VB. Then, the excitation process 
leaves behind a hole in the VB [19]. Here, as a net result, electrons (e7) and holes 
(h*) pair are generated by the Eq. (1) below. 


TiO + hv(UV) — TiO2(e~ (CB)) + TiO2 (ht (VB)) (1) 


Afterward, the photogenerated holes at the VB react with a water molecule to generate 
hydroxyl (OH") radicals. These hydroxyl radicals are extremely powerful oxidizing 
agents, which attract adsorbed organic molecules to mineralize them depending upon 
their structure and stability level [19]. While the photogenerated holes react with 
water to generate the hydroxyl radical, at the same time, the electrons in the CB are 
taken up by oxygen (O2) molecules to produce anionic superoxide radical (O2~*). 
These superoxide radicals are formed into hydroperoxyl (HO) radicals and the 
subsequent hydrogen peroxide (H202), which further dissociates into highly reactive 
hydroxyl (OH ) radicals. All these oxidation and reduction processes commonly take 
place on the surface of the TiO, photocatalyst [19]. Therefore, the surface area of 
the TiO, is highly important to achieve the high photocatalytic activity for desired 
application. 

Recent studies have been revealed that the particle size of TiO, plays a crucial 
role in photocatalytic activity [20]. Decreasing the particle size to the nanoscale 
(increasing active surface area) leads to a higher photocatalytic activity. However, 
the conventional nano-powdered photocatalysts need post-treatment separation in a 
slurry system after the photocatalytic reaction. This problem can be overcome by 
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immobilizing TiO% particles as a robust and stable thin film on solid surfaces [21, 
22]. However, thin films have a limited surface area in comparison to nanoparticle 
systems and they show limited photocatalytic activity, which has to be improved 
by further strategies such as (i) tailoring morphology and crystal structure, (ii) 
decorating/depositing noble metals (Au, Ag, Pt, etc.) and (iii) coupling metal oxide 
semiconductors (ZnO, CeOs, etc.) [23]. 

Noble metal nanoparticles (Au, Ag, Pt, etc.) deposited on semiconductor surfaces 
have been extensively studied due to their unusual (photo)catalytic and optoelec- 
tronic properties [24] as well as their potential to tailor the sensor properties of 
semiconducting metal oxide gas sensors 11. In particular, TiO2 decorated by noble 
metal nanoparticles like Au and Ag has received more and more attention because of 
its significant improvement in the photocatalytic activity of TiO, for specific appli- 
cations [22]. The size and distribution of these noble metal nanoparticles are the 
most effective parameters on the activity of photocatalyst [24]. Additionally, the 
photocatalytic performance of TiO2 modified by noble metals highly depends on 
some external factors such as strong contact with metal and support [25]. Noble 
metals can be deposited on TiO2 surface by various methods including sputtering, 
physical mixing, chemical reduction, electrodeposition, photocatalytic reduction, 
and so on. However, among the mentioned methods, the photocatalytic deposition 
(sometimes it is called photodeposition or photoreduction) is a facile, simple, and 
low-temperature process with efficient metal-support interaction especially for TiO2 
thin film as support [26]. 

The photocatalytic deposition method is based on the photocatalytic properties 
(bandgap, activity, etc.) of semiconductor materials. Generally, several conditions 
are needed to allow the deposition of metals on the semiconductor surface. First, 
the photon energy of the light (solar or artificial light) should be larger than the 
energy bandgap of the semiconductor. Mostly, photocatalytic deposition of metal to 
TiO, employs under artificial ultra-violet (UV) light illumination due to TiO2’s wide 
band gap, which is around 3.0 eV. When the light has proper energy, the light is 
absorbed by a semiconductor, and electrons in its VB are excited to its CB. These 
electrons are used for the reduction process. Second, the CB energy level of the 
semiconductor should be more negative than the reduction potential of the metal ion 
(M+). Third, the efficient charge (e~ and h*) separation and migration, by electron 
donor, is necessary to continue the reaction [27]. Finally, the semiconductor acts as 
a template for metallic structures as shown in Fig. 1. 

The depositing of noble metals on semiconductor surface by photocatalytic 
deposition is possible, if all conditions, which are mentioned above, are avail- 
able. However, various parameters are also needed to be considered to allow the 
well-controlled (precise shape, size, and distribution) photodeposition process to 
occur on semiconductor surfaces such as sacrificial reagent, pH, temperature, metal 
precursor, light exposure time and intensity, absence, or presence of oxygen in the 
media [14, 28, 29]. 

In this chapter, photocatalytic deposition is discussed as an approach to form 
metallic structures from a global liquid reservoir. In this context, the photocatalytic 


Photocatalytic Deposition for Metal Line Formation 245 


Fig. 1 Schematic M“ 
illustration for 
photodeposition of metal 
structures on semiconductor 
surface (M: Metal and D: 
Electron donor) 


light 


a p™ 


D 


properties of TiO}, thin films are employed to reduce metallic species from the sur- 
rounding solution. This chapter will elucidate the fundamental process of photo- 
reduction with photocatalytic TiO% thin films and will showcase the applicability 
towards the formation of metallic structures at the example of arrangements of locally 
grown metallic Au structures. 


2 Reactive Sputtering of Photocatalytic TiO. Thin Films 


TiO; is typically characterized as a n-type semiconductor and has three crystalline 
phases such as anatase (tetragonal), rutile (tetragonal), and brookite (orthorhombic) 
[30]. In general, anatase shows higher photocatalytic activity than rutile and brookite 
due to mainly its indirect bandgap structure (low electron-hole recombination rate). 
Therefore, most of the researchers focused on preparing of mainly anatase TiO) struc- 
tures for achieving high photocatalytic activity [31]. However, recent studies have 
shown that the mixture of anatase and rutile has a synergetic effect on decreasing the 
recombination rate of photogenerated electron-hole pairs, which boosts the overall 
photocatalytic performance [32]. 

As mentioned before, TiO» is extensively employed in various applications due to 
its unique optical, electronic, and photocatalytic properties. In these applications, it 
is mainly used as powder form (nanoparticles around 25 nm size), which shows high 
photocatalytic activity results from the high surface area. However, the separation 
of these tiny particles from the reaction media after the end of the reaction is highly 
difficult with conventional filtration systems [33]. Recently, TiO. thin films have 
gradually replaced conventional TiO, powders due to separation and agglomeration 
problems, especially for their long-term use. 

Various deposition methods have been utilized to prepare TiO, thin films such as 
sol-gel dip coating, sol-gel spin coating, chemical vapor deposition (CVD), phys- 
ical vapor deposition (PVD), electron beam, spray pyrolysis method [34]. Among 
these deposition methods, PVD methods have been extensively used because they 
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can easily control the composition and morphology of the prepared thin film. Various 
PVD methods are being utilized for the deposition of TiO. thin film such as thermal 
evaporation, pulsed laser deposition, DC, and RF magnetron sputtering [35]. DC 
magnetron sputtering is one of the most preferred methods because of its good 
adhesion, high deposition rate, high quality, and uniform film [36]. In this deposi- 
tion method, there are many vital parameters (sputtering power, distance between 
substrate and target, argon/oxygen ratio, working pressure, substrate temperature, 
and annealing treatment) to optimize the quality of the thin film. However, it is still 
a challenge to achieve highly photocatalytic active TiO} thin films, which compete 
with TiO, nanoparticles, due to their limited surface area [33]. To achieve a high 
surface area, nowadays an effective process is established, which is called as nano- 
crack network formation, within sputter-deposited TiO, thin films. For example, 
Henkel et al. reported on thermally controlled nano-crack formation as a possible 
method to improve the photocatalytic performance of well adhering, reactively DC 
sputtered TiO, thin film [37]. Networks of nanoscopic cracks were produced into 
tailored columnar TiO) thin films by thermal annealing as shown in Fig. 2. 

It can be easily seen from Fig. 2a, irregular and narrow columns (underlined by 
blue and red marks, respectively) dominate the formation of the film at the TiO2- 
substrate interface. According to the author’s observations, broader columnar struc- 
tures start to grow at a distance of 200 nm from the substrate and become more 
dominant features at TiO» thin film surface. However, after thermal annealing, the 
deep trenches separate into small bundles of TiO; columns, which enhance the overall 


(a) before tempering (b) after tempering (650°C, 1h) 
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Fig. 2 SEM images of sputtered TiO thin films, a before and b after thermal annealing (at 
650 °C for 1 h), important morphological features are marked by red and blue lines. (Printed 
with permission from [37]. Copyright (2018) IOP) 


Photocatalytic Deposition for Metal Line Formation 247 


photocatalytically active surface area (Fig. 2b). Additionally, they also reported that 
the thermal annealing step (at 650 °C for 1 h) is highly crucial not only for the 
formation of the nano-crack network but also the transformation of amorph phase 
(as-deposited TiO.) to the mixture of anatase and rutile phase (anatase is more domi- 
nant), which is extremely beneficial for enhancing photocatalytic activity. Similarly, 
Zhan et al. observed that the correlation between annealing temperature and phase 
transformation of TiO, from amorph phase to anatase (>300 °C), partially rutile 
(around 600-800 °C), and fully rutile phase (at 1000 °C) [34, 38]. 

Nano-crack network formation is a promising approach to achieve a high surface 
on the thin film surface. It can be one of the biggest features on competition between 
thin film and nanoparticles photocatalyst. Ghori et al. demonstrated that sputter- 
deposited TiO» film with nano-crack networks shows extremely high photocatalytic 
activity in comparison to a reference TiO, thin film prepared by immobilized TiO, 
(Degussa P25) nanoparticles [21]. Here, the authors reported that sputtering of TiO2 
at high oxygen partial pressure and low deposition rate leads to the formation of high 
aspect ratio structures with weak inter-columnar bonding, which can be transformed 
to a crack network by thermal annealing process (Fig. 3a). 

As a comparison, they prepared the reference thin film by spin-coating. The 
prepared thin film has a porous structure composed of commercial spherical TiO, 
nanoparticles, which are well distributed and maintained their original size (25- 
30 nm) and shape (Fig. 3b). The thickness of the sputter-deposited and the reference 
TiO, thin films are around 400—450 nm for a reliable comparison. The photocatalytic 
activity of both TiO, thin films was investigated by degradation of methylene blue 
(MB) aqueous solution under UV illumination. The results showed that the sputter 
deposited TiO» film degrades 90% of MB in 4.95 min while the reference TiO, thin 
film needs about 25 min (Fig. 3c). 
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Fig. 3. SEM images of a sputter deposited TiO? film after thermal annealing and b spin-coated 
reference TiO? film. (Insets show cross-sectional SEM micrographs) c Normalized degradation 
(plot) of MB solution in the presence of sputter-deposited and reference TiO» thin films under 
UV illumination. (Quartz substrate (given as blank) is used as the control substrate). (Printed with 
permission from [21]. Copyright (2018) Elsevier) 
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The thermal annealing process is an easy and reliable process for phase trans- 
formation and formation of a nano-crack network for better photocatalytic perfor- 
mance as aforementioned. However, some additional process parameters can be also 
controlled for high photocatalytic performance. Generally, crystalline TiO» thin film 
is deposited on the substrate at low operating temperatures. Here, the crystallinity 
of TiO, thin film can be mainly tuned by changing total pressure and oxygen partial 
pressure [39]. The total gas pressure affects the kinetic energy of sputtered atoms, 
which increases the probability of collisions and acceleration of particles and subse- 
quently the particle energy [40]. Therefore, the anatase phase forms at high total 
pressure when the rutile phase forms at low total pressure [40, 41]. 

During magnetron sputtering, argon (Ar) and oxygen (O2) are used as the plasma 
gas and the reactive gas, respectively. Oxygen plays a significant role in the deposition 
of TiO, thin film via magnetron sputtering especially for reactive sputtering. Many 
researchers have pointed out that the oxygen concentration during the magnetron 
sputtering might affect the formation of oxygen vacancies in the deposited thin 
film [41]. Additionally, the oxygen partial pressure (the oxygen/argon ratio) has a 
huge effect on the discharge parameters, such as plasma potential, discharge voltage, 
and ion composition of discharge [42]. For example, Zhang et al. reported that the 
discharge voltage increases with increasing oxygen flow rate up to a certain threshold. 
Before the threshold is reached, a mixture of metallic Ti and TiO is deposited on the 
substrate. The uniform TiO; thin film forms on the substrate only after the oxygen 
flow rate reaches this certain threshold [34, 43]. 

Although numerous attempts have been performed to investigate the relationship 
between the oxygen/argon ratio and photocatalytic activity of TiO, thin film, no clear 
agreement has been reached so far. However, a lot of properties of TiO, thin film are 
affected by oxygen/argon ratio such as deposition rate, grain size and surface rough- 
ness, surface chemical composition, and optical parameters, which might directly 
or indirectly influence the photocatalytic activity [44, 45]. For example, Vahl et al. 
reported the availability of oxygen during reactive sputtering is a vital parameter for 
tailoring of thin film morphology for high photocatalytic performance (Fig. 4) [46]. 
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Fig. 4 Top-view SEM images of TiO? thin films for different oxygen/argon flow ratios, before 
(top row) and after (bottom row) thermal treatment (Printed with permission from [46]. Copyright 
(2019) IOP) 
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The authors reported that the formation of nano-crack networks is more noticeable 
for TiO, thin films deposited at higher oxygen/argon ratios. The reason for this might 
be the presence of crystalline anatase seeds for high oxygen partial flows. These seeds 
grow by thermal annealing process more effectively as a well-defined nano-crack 
network. Here, the annealed TiO; thin films deposited different oxygen/argon ratios 
showed different activity on degradation of MB under UV illumination. After 7 h 
UV illumination, the TiO, thin film deposited at 3% oxygen/argon ratio degraded 
almost 81.2% of MB when this value is 96.1% for TiO, thin film deposited at 6% 
oxygen/argon ratio. 


3 Deposition of Metal Structures from Photocatalytic 
Reduction 


As discussed in the previous chapters, various strategies have been proposed to 
improve the volume to surface area ratio of TiO, thin films for a high photocatalytic 
activity. Additionally, TiO. thin film incorporated with noble metal nanoparticles can 
be considered another strategy to enhance the (photo)catalytic properties by means of 
tuning the Fermi level of TiO, and acting as an electron sink [47]. The prepared noble 
metal/TiO, hybrid structure has a huge potential on not only environmental applica- 
tions but also different kinds of applications including hydrophobic and hydrophilic 
coatings, sensor devices, reconfigurable connections, etc. [13]. 

In the literature, there are numerous methods, which can be used to synthesize the 
noble metal/semiconductor nanocomposites such as hydrothermal, droplet, precipi- 
tation, electrochemical methods, and so on [27]. The photodeposition is one of the 
simplest methods to prepare the noble/semiconductor hybrid structure by using only 
the bandgap of the semiconductor [48]. This method needs only the irradiation of a 
light source (artificial or sunlight) rather than requiring additional reducing agents, 
high temperature, multi-step processing, etc. [49]. Furthermore, the photodeposi- 
tion method is easily applicable to 2D substrates (such as thin film) by adjusting 
the concentration of the (noble)metal precursors, irradiation intensity, and dura- 
tion [50]. For example, Mendoza-Diaz et al. demonstrated that Au nanoparticles 
were deposited on columnar TiO» structure by photodeposition process under UV 
irradiation (365 nm, 100 W, 30 min) by using an Au precursor solution (Fig. 5) [51]. 

It can be easily seen from the SEM images in Fig. 5a, the TiO, thin film 
has a columnar morphology with grain sizes from ~20 to 50 nm. This columnar 
morphology is generally observed in TiO, films, which are deposited by chem- 
ical and physical vapor deposition techniques [52, 53]. For the Au-on-TiOz sample, 
Au nanoparticles were deposited on top of the TiO, thin film surface (Fig. 5b). 
The photodeposited Au nanoparticles have a spherical shape with a size range of 
30—70 nm (inset in Fig. 5b). The authors reported that the size dispersity of the 
photodeposited nanoparticles is closely related to the geometrical restraints enforced 
by the roughness of the columnar TiO}, structure [51]. 
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Fig. 5 (Left) Schematic illustrations of TiO2 and TiO2/Au hybrid structures. (Middle) SEM images 
of the sample surface area. (Right) HAADF-STEM cross-sectional images: a TiO}? thin film; b Au- 
on-TiO2 (inset: Au NP size dispersion in yellow). Letter C corresponds to the top layer of protec- 
tive sputtered carbon. (Printed with permission from [51]. Copyright (2020) American Chemical 
Society) 


Similarly, Veziroglu et al. showed that not only the TiO2 morphology but also the 
composition of the precursor solution has a significant effect on the size and distri- 
bution of the photodeposited Au structure as demonstrated in Fig. 6 [28]. Figure 6a 
shows that only a few Au nanoclusters (surface coverage of 9.2%) were photode- 
posited on TiO, surface water was used as a solvent. However, densely distributed 
Au nanoclusters (surface coverage of 51.6%) on TiO, were obtained when a mixture 
of 1-hexanol—water (v/v: 20/ 80) was used (Fig. 6d). 

Here, one can easily see differences in morphologies given in helium ion 
microscopy (HIM) images (Fig. 6a—d), the solvent type (or composition of the 
precursor solution) significantly affects the final morphology of the photodeposited 
Au nanoclusters. Because, during the photodeposition process, the TiO» thin film 
contact with the Au precursor solution, which forms the solid—liquid interface. This 
interface plays a crucial role in the electron transferring from a photocatalyst surface 
to electron-acceptor species (e.g., Au** ions) in the solution. When the composition 
of the precursor solution changes, the distribution of the electrical potential also 
changes [54]. This phenomenon creates various pathways to reduce Au** ions on 
TiO, as metallic Au nanoclusters, which form different shapes, sizes, and distribution 
profiles. 

Similarly, the pH of the precursor solution also affects the charge distribution in 
the solid—liquid interface. Unfortunately, according to the author’s best knowledge, 
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Fig. 6 Helium ion microscopy (HIM) images of photodeposited Au nanoclusters onto TiO? thin 
film surface in a water, b acetone—water (v/v: 20/80), c isopropanol—water (v/v: 20/80), and d 1- 
hexanol—water (v/v: 20/80) mixtures. e Surface coverage (%) of photodeposited Au nanoclusters 
with different solvents. (Printed with permission from [28]. Copyright (2020) American Chemical 
Society...) 


there is not any systematic research related direct effect of the pH on photodeposited 
Au structure especially on TiO, thin film surface, yet. However, similar studies can 
give us an overall understanding of the pH effect on the photodeposited particles. 
For example, Guo et al. reported that pH plays a significant role in the nucleation 
position of the photodeposited Au nanoparticles on to plate-like BiOBr [55]. It can 
be seen in Fig. 7, SEM images after the photodeposition process show facet-selective 
deposition of Au particles with pH-dependency. At pH 3, they observed an overall 
deposition of Au nanoparticles, favorably on the top ([001] facet) as shown in Fig. 7a. 
However, when increasing the pH of the precursor solution to 3, 5, and 9, the pattern 
of the photodeposited nanoparticles gradually changes (Fig. 7a—c). For example, Au 
nanoparticles are founded on every facet ([001] and [102], top and side, respectively) 
at pH 5 while only on the side ([102] facet] at pH 9. 


Fig. 7 SEM images of BiOBr showing pH-dependent geometrical deposition of Au nanoparticles. 
Au/BiOBr at a pH 3; b pH 5; c€ pH 9. The scale bar always corresponds to 200 nm. (Printed with 
permission from [55]. Copyright (2018) Royal Society of Chemistry) 
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The authors reported that only at a low pH value, the charge density of the semi- 
conductor and precursor interface is weak. Therefore, [001] facets are weakly posi- 
tively charged, which might significantly reduce the recombination of the photoin- 
duced charge carriers (electron and holes). This phenomenon may enhance the elec- 
tron/ hole mediated reaction on the semiconductor surface during the photodeposition 
process. However, at a high pH value, all facets are rather strongly negatively charged. 
Therefore, electrons are no longer attracted towards the [001] facet surface. They are 
no longer available for the reduction of Au ions on the surface. Hence, the facet- 
selectivity of the photodeposition reaction of Au ions to metallic Au nanoparticles 
reverses from the [001] facet to [102] facet at a high pH value [55]. 

As already mentioned, the composition and pH of the precursor solution have a 
huge effect on the size, distribution, and position of the photodeposited nanoparticles 
on the semiconductor surface due to various charge distributions in the solid—liquid 
interface during the photodeposition process. Additionally, the photocatalytic activity 
of the semiconductor affects the final morphology of the deposited particles on the 
surface because it limits how many electrons can be generated by the photocatalyst 
under light illumination. For instance, Veziroglu et al. demonstrated that hierarchical 
Au needle clusters (HAUNCs) were deposited on a highly active TiO, thin film 
surface via UV illumination (Fig. 8a—b). Here, the size and the geometry of deposited 
HAuNCs were controlled by simply altering the photocatalytic activity of the TiO2 
(depends on the crystal structure), UV light intensity, and irradiation time [14]. The 
basic growth mechanism of a HAuNC is semantically shown in Fig. 8c. First, the 
photoinduced charge carriers (electrons and holes) in TiO» thin film are generated by 
UV illumination. These electrons reduce the Au** ions into a stable Au cluster. This 
first step is almost same with all conventional photodeposition process for different 
kind of metallic structures [56]. However, a high density of electrons can be generated 
by the prepared highly photocatalytic active TiO» thin film. Therefore, the relative 
decrease of Au** ions on the surface, where the first embryonic Au cluster nucleate, 
occurs in a very short time. This promotes the diffusion of more Au** ions (from 
higher to lower concentration regions), which is followed by preferential piling of 
Au nanoclusters into needle-like structures (Fig. 8a—b). It seems that the high density 
of electrons triggers the directional growth of Au nanostructures (Growth). These 
dense electrons, which are generated by TiO thin film, can be trapped by former Au 
nanoclusters (nuclei) and they act as the further nucleation sites for incoming Au** 
ions from the precursor solution (Needle formation). 

Additionally, the authors reported that UV light intensity plays a major role in 
the final. size and sharpness of the deposited Au structures during the deposition 
process. Because the UV light intensity also decide the number of photogenerated 
electrons by TiO2, which leads to quick nucleation and a high growth rate. This is a 
piece of key information about controlling the nucleation point during the photode- 
position process. For this purpose, they used the polymer mask between the light 
source and the sample. Here, patterned HAuNC structures on TiO» thin film surface 
were obtained by selective light illumination as shown in Fig. 8d—e. These patterned 
HAuNC structures find some applications especially for catalysis, plasmonic, and 
biomedical technologies. 
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Fig. 8 a SEM image of HAuNCs deposited on TiO. b HIM image of a single HAUNC. ¢ Schematic 
representation of HAuNC growth mechanism. SEM images of d periodic HAuNC arrays and e a 
single HAuNC. (Printed with permission from [14]. Copyright (2020) Wiley) 


4 Metal Line Formation by Photocatalytic Reduction 


As discussed in the previous section, localized irradiation of a continuous TiO, thin 
film with the use of a shadow mask results in the formation of metallic nano- and 
microstructures from the precursor solution. In the following, a second approach to 
obtain localized formation of metallic structures will be discussed. In essence, this 
approach is based on the image reversal lithography process. The general process 
scheme is depicted in Fig. 9. The process is similar to the positive resist lithography 
except that in this case, a chemical modification step is included to ensure the cross 
linking of the resist. Also, the process involves a double exposure of the resist which 
causes the positive resist to act like a high-resolution negative resist. 

The procedure involves: cleaning and priming of the glass wafer, spin coating of 
the resist, soft baking (110 °C for 50 s), exposure, hard bake (120 °C for 2 min), 
flood exposure, development, inspection, TiO, deposition, lift off, final inspection. 
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Fig. 9 An illustration of the major steps in the image reversal process necessary to obtain structures 
with high resolution 


The image reversal technique is preferred because the idea is to have the reversed 
structure of the mask transferred on to the glass substrate. The TiO2 which is then 
later deposited directly on to the glass surface takes the pattern of the mask while 
everywhere else is covered with resist. After the lift off (using acetone), only the 
structured TiO, (which is unaffected by the acetone) remains on the substrate. With 
this approach, very thin lines of TiO are transferred to the glass substrate. It allows 
for fabrication of quite small structures on a substrate. For example, TiO}, lines of a 
few microns in width is structured on a glass substrate with sufficient accuracy. 

The substrates with lithographically structured TiO, thin films are consecutively 
used as substrates for photocatalytic deposition of metallic gold from precursor solu- 
tion. For in-situ monitoring, the absorption and reflection of the photocatalytically 
grown gold on the titanium dioxide surface is taken as a measure. A standard transmis- 
sion microscope (Leica DMi8) setup is used with an external UV-LED to stimulate 
the photocatalytic process as shown in Fig. 10. 

A spectrograph extension of the setup is used to record the bright-field and UV- 
LED spectra. The UV-LED has its peak at approximately 365 nm, while the bright 
field source has no energy in the UV regime and is thus suitable for the in-situ 
monitoring. The intensity of the UV-light is controlled by the distance to the sample 
and was determined with a power meter (Newport 2936-C) for different distances. 
Samples are positioned in a beaker filled with HAuCl, solution. Images are taken in 
regular intervals and are computed to obtain growth dynamic graphs of the process. 
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Fig. 10 Transmission microscope setup for in-situ monitoring of photocatalytic gold growth. 
Photocatalytic gold growth is excited by an external UV-LED. A standard microscope setup (Leica 
DmI8) with bright-field illumination is used to track the growth by recording images during the 
growth process 


In a first experiment Au is grown using a 0.736 107° mol/l HAuCl4 solution on 
a sample covered partially with a titanium dioxide layer for different illumination 
intervals. Figure 11 shows three example transmission images of the sample. On the 
left side there is no TiO2, while the right side is covered with titanium dioxide. Due 
to photocatalytic Au growth on the TiO; the transmission intensity is reduced. The 
total normalized transmission was 0.72 after completion of the total duration of the 
experiment. 

Within the MATLAB environment different areas of the images are selected and 
evaluated over time. First the RGB channels as well as all pixels are summed up for 
these areas and are subsequently normalized to the first pictures, which are regarded as 
full normalized transmission for every measurement. The normalization accounts for 
different loss mechanisms such as thin-film interferences, absorption and scattering 
in the solution and the beaker. In the following, the three red reference areas and 
the three blue areas on the titanium dioxide are considered in more detail. The time 
evolution of the transmission for these six areas is depicted in Fig. 12. Time intervals 
of two minutes without illumination are followed by five minutes of UV illumination. 
Two illumination levels are employed-a lower excitation intensity of 1.8 mW/cm? for 
the first two intervals and a higher intensity of 2.7 mW/cm/? for the third illumination 
phase. 
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Fig. 11 Bright-field microscope transmission images at three different times during photocatalytic 
gold growth for a sample half-covered with TiO2. The computation areas are depicted in the left 
picture. In the red reference areas, there is no TiO2 and negligible growth occurs. The blue areas 
are on the TiO? and become darker as the gold layer on the surface increases during growth. The 
normalized transmission on the titanium dioxide goes down to 0.72 after two hours 
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Fig. 12 Normalized transmission measurement of gold growth dynamics on titanium dioxide 
surface with different illumination powers. The image areas corresponding to the six timelines are 
indicated in Fig. 11. The blue lines show the transmission for areas on the titanium dioxide surface, 
the red lines are reference areas on the substrate. Different illumination intervals are depicted 


The results presented in Fig. 12 clearly show the suitability of the chosen method 
for monitoring the gold growth process. In the reference areas (red lines) almost no 
intensity change is observed indicating that no gold is deposited on the substrate. In 
the three areas on the titanium dioxide (blue lines) the change in transmission intensity 
is clearly visible. Furthermore, it is observed that the growth speed is controlled by 
the illumination intensity. 

Figure 13 shows the growth results on a TiO. line with 200 um width. Three 
growth processes were conducted as depicted from stages (a-d). The first growth 
experiment (0.736 107? mol/l HAuCly, 2.7 mW/cm? UV intensity, 21 h) yielded only 
gold clusters and no full coverage. Repeating the experiment led to a full coverage 
of the surface, which was then destroyed by drying the sample with nitrogen as seen 
in Fig. 13c. To have full coverage the sample underwent the growth experiment a 
fourth time, showing fast and homogeneous growth over the titanium structure. As 
shown in the graph and Fig. 13d. We found that the growth process is faster and has a 
better coverage, when the growth is done repeatedly. This effect was also observable 
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in other experiments within the same setup and is currently under investigation. A 
picture of the grown gold lines can be seen in Fig. 14. The linewidth varies from 10 
to 500 um. 

Figure 15 shows the results for grown metal lines with different line widths, 
ranging from 50 to 250 um. The growth showed to be homogenous. Damages within 
the lines resulted from drying the sample. 

The samples were further characterized after the growth process with scanning 
electron microscopy (SEM) to get a better understanding of the layer composition 
and the coverage of the TiO? with Au particles. After the first growth in HAuCl4 
solution, sparsely distributed Au nanoclusters are observed with large spaces between 
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Fig. 13 Different stages of gold growth on a 200 um TiO} line. a shows the structure before growth, 
b, c and d after the first, second and third growth experiments. The graph depicts the normalized 
transmission measurement for the growth process between (c) and (d). The green lines are off 
structure reference spots, blue is on the connector dots (here to be seen just the left one) and red is 
on the line itself 


Fig. 14 Reflection 
microscopy image of the 
grown Au lines with varying 
widths and lengths 
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Fig. 15 Gold growth on titanium dioxide lines with different line widths: 50 um (a, f), 100 ym 
(b, g), 150 um (c, h), 200 um (d, i), 250 um (e, j) 


the growing clusters (compare Fig. 16a). An explanation for this observation could 
be the rate of the electron transfer at the TiO2—solvent interface [28]. The rate of 
electron transfer is dependent on the potential gradient at the interface which could 
be inversely proportional to the dielectric constant of the liquid. For water, which 
has a high dielectric constant, a lower electron transfer rate is expected [28]. 

After a second and third growth process on the same sample, the surface coverage 
of the Au nanoclusters increased drastically. It is possible that already deposited Au 
nanoparticles act as nucleation sites for aggregation, and the nanocluster will tend 
to grow in the more energetically favorable direction. The SEM image in Fig. 16b 
shows that large clusters have formed compared to the first growth, supporting the 
hypothesis. 

It shows additionally that photocatalytic deposition of Au** still occurs on the 
surface without a nucleation source, leading to new formations of clusters. Their 
growth depends on how much of the TiO,—layer can be still excited by the UV- 
light despite the presence of Au particles. They reflect the light, which prohibits the 
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Fig. 16 SEM-images with 8000 x times magnification of TiO2 with photocatalytic growth of Au 
clusters one time (a) and three times successively (b) 
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generation of electrons in that area. The experiment shows that the surface coverage 
can be improved a lot by successive photocatalytic gold growth. 


5 Conclusion 


In this chapter the photodeposition of metallic nano- and microstructures is show- 
cased at the example of locally grown metallic Au structures on photocatalytically 
active TiO, thin films. Among the various deposition methods for TiO» thin films, 
ranging from sol gel synthesis over evaporation to sputtering, in particular reactive 
DC magnetron sputter deposition has been found to offer the potential to tailor the 
morphology of the photocatalytic thin films. At the example of introducing nano- 
crack networks upon heat treatment it is shown how photocatalytic performance of 
thin films can be tailored. 

Using photocatalytic TiO, thin films as a substrate, metal structures such as Au 
micro- and nanoparticles can be deposited on the thin film surface via a photore- 
duction reaction. Upon the illumination of TiO2 thin films with UV light, the metal 
precursor is reduced via reaction with photogenerated electrons/holes, which results 
in the formation of solid metal structures. The morphology, size and coverage of 
the photodeposited metal structures can be greatly influenced by the choice of TiO 
substrate, illumination intensity and the composition of the precursor solution. With 
high photocatalytic performances of the TiO, thin film it is possible to obtain high 
aspect ratio structures such as nanoneedles. The choice of thin film as well as the 
illumination intensity determines the morphology of the obtained structures. Lateral 
selectivity in the deposition of the metallic structures was showcased at two exam- 
ples: On the one hand, selective illumination (e.g., by using shadow mask) results in 
localized gold growth. On the other hand, by structuring the TiO} thin film (e.g., via 
lithography and lift-off) the formation of metal structures can be locally restricted. To 
obtain better insight and control, the formation of metal structures via UV-stimulated 
photodeposition can be monitored in-situ via changes in the transmission. The density 
of the grown nano- and microstructures can be varied upon performing consecutive 
growth steps from individual nanostructures towards a dense coverage. In addition, 
the surface coverage can be tailored further upon adding additives to the precursor 
solution. As such, light-stimulated photodeposition of metal structures is an inter- 
esting technique for the local on-demand formation of metallic aggregates and has the 
potential to contribute towards an inclusion of long-range connections and globally 
mediated aspects like homeoplasticity into the field of neuromorphic systems. 
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Abstract Sensors play a crucial role in our everyday life and will become more and 
more demanded with the transition towards smart cities and the “Internet of Things” 
with the result of an ever-increasing energy demand. Thus, the research of improving 
the energy efficiency of sensor systems has started to move towards edge computing 
and neuromorphic engineering while at the same time additive manufacturing has 
gained increasing attention as a means for a rapid, scalable fabrication of functional 
devices yet with huge design freedom and quick iteration cycles during the devel- 
opment phase. In this chapter the junction of pre-designed components with self- 
organizing material systems for the facile fabrication of sensor devices via direct ink 
writing is demonstrated. As examples 3D-printed CuO/Cu20/Cu- and CuO/Fe203 
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acetone gas sensors are shown, as well as how the decoration of metal-oxide semi- 
conducting sensor materials with noble metallic nanoparticles can enhance the sensor 
properties with respect to sensitivity, selectivity and stability. Finally, the assembly 
of conductive bridges between electrodes stimulated by voltage pulses is introduced 
as an approach towards facilitating the self-organization of neuromorphic circuits. 


Keywords Memsensors - Metal-oxide semiconductor sensors - 3D-printing - 
Direct ink writing - Triggered assembly 


1 Introduction 


Sensors are a key component of our everyday life. They are used to detect changes 
in the environment that are hazardous or of interest otherwise. Living organisms 
use environmental stimuli triggering their sensory systems as inputs to adapt their 
behaviour to the environmental surrounding, e.g., for finding food or evaluating dan- 
ger. In this regard biological systems have been evolutionary optimized to be most 
energy-efficient since this is a crucial advantage for survival. An example is the adap- 
tation to a permanent background stimulus [1]. The sensory system changes such, 
that the response to an excitation decreases, if the stimulus does not change over a 
long period of time. For example, a human eye in a dark room contracts its pupil 
when suddenly the light is turned on, accommodating to the new lighting situation, 
thus allowing the eye to see normally after adaptation. However, technical advances 
presented humanity with new challenges surpassing the capacities of humans’ nat- 
ural sensors. Technical sensors have been developed to compensate or extend the 
capabilities of the human body, especially for security and health applications e.g. 
monitoring the concentration of harmful radiation or toxic gases, but also for a wide 
variety of purposes like magnetic field sensors, CCD-sensors or mechanical sensors. 
With the ever growing number of electronic devices and the development towards the 
“Internet of Things” and “Smart Cities” the number of deployed sensors will increase 
significantly in the future and a crucial aspect is the growing energy demand, that 
accompanies this development [2—4]. However, biological systems are remarkably 
efficient, when it comes to data acquisition and processing. This has inspired research 
to mimic the way they achieve their great energy efficiency. A prominent approach 
for higher energy efficiency to account for the growing energy demand is overcoming 
the von-Neumann bottleneck by using features of edge computing and neuromorphic 
engineering, i.e. decentralizing computing logic and memory to reduce the power 
consumption of communication between them and diminish the need for data trans- 
mission to a central server [5-8]. In the following sections, various concepts will be 
discussed, that advance the research in sensor technology to account for the demand 
of efficient, scalable, specialized and smart sensor devices. At first the concept of the 
memsensor will be introduced, which is a novel junction of a sensor with memristive 
components yielding additional unique abilities, such as adaptation to a permanent 
background stimulus. It will be discussed with respect to an equivalent circuit model 
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and a ZnO needle as an example system. When it comes to the fabrication of sen- 
sor devices, most processes rely on micro-patterning by lithography and physical or 
chemical vapour deposition techniques, which are inherently demanding with respect 
to workplace and material quality [9, 10]. Here, additive manufacturing (AM), which 
is based on inexpensive and highly available equipment, is discussed as a means of 
fabrication for sensor devices. The AM approach provides huge design freedom with 
respect to sensor materials and shape as well as quick iteration cycles making it a 
valuable tool for research and development of sensor devices. Metal-oxide semicon- 
ductor (MOS) microparticles are showcased as sensor materials printable by direct 
ink writing (DIW) and the enhancement of their sensor performance by decoration 
with metallic nanoparticles is discussed. In the last section, the dynamic establish- 
ment of conductive interconnections between electrodes at the microscale in a liquid 
matrix is introduced by the example of copper electrodes in an electrolyte of dimethyl] 
sulfoxide (DMSO) and methylene blue (MB), which serves as a first step towards 
expanding brain-inspired plasticity mechanisms to a larger scale. 


2 Smart Sensors 


This chapter evolves around sensor devices that belong to the category of semi- 
conducting metal oxide micro- and nanostructures, such as CuO, TiO2, ZnO. Such 
materials are commonly used for sensing applications, for example as photon sen- 
sors or gas sensors. While photon sensing involves a considerable fraction of the 
material, depending on the photon penetration depth in the material, gas sensing is 
strongly related to the adsorption and desorption of molecular species onto the sen- 
sor surface and therefore can be considered as a surface phenomenon. Interestingly, 
there is a considerable overlap between materials used for MOS sensors and mem- 
ristive devices that rely on the valence change mechanism upon migration of oxygen 
vacancies. This implies, that if manufactured in a proper way, a single device can 
intrinsically unify the inherent functionalities of sensors and memristive devices, 
namely the stimulus dependent resistance and the controllable switching between 
resistance states. This section will elucidate the properties of memsensors that arise 
upon the junction between memristive switching and stimulus sensing. In order to 
obtain memsensitive properties, an important requirement is that the active material 
must be accessible to the chosen external stimuli. This requirement strongly depends 
on the type of stimulus—while gas sensors need a direct contact of the gas species to 
the sensor surface, other stimuli like photons or magnetic fields may also potentially 
penetrate through matter. This requires the top electrodes of vertically sandwiched 
devices to be chosen carefully or to be additionally structured. Another option is the 
use of lateral structures, like micro- or nanowires that are in direct contact with the 
surrounding medium. An example for such horizontal structure are ZnO micronee- 
dles. A ZnO needle with a diameter in the um range and a length in the cm range is 
shown in Fig. la. Details on the manufacturing are reported elsewhere [11]. 
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Fig. 1 A ZnO microrod device a as a memsensor device prototype is capable of showing stimulus 
dependent hysteresis. b An equivalent circuit ¢ with three components was used to model stimulus 
dependent hysteresis d and adaptation of the resistance of amemsensor device to an external stimulus 
e. In case of amplitude adaptation e, the resistance response (black) of the modelled memsensor 
decreases with each consecutive stimulus pulse decreases, which is based on the slower change in 
the internal state of the parallel resistor (red) than the serial resistor (green) [11]. Reprinted with 
permission from [11] 


The electrical resistance of a ZnO microneedle exhibits a strong dependence on 
the illumination by UV photons. Interestingly, IV hysteresis measurements of the 
ZnO needle device show an UV dependent hysteresis, which is depicted in Fig. 1b. 
While the IV characteristics in the dark state do not show a pinched hysteresis and 
in general exhibit an overall very high resistance, under UV illumination there is a 
pronounced pinched hysteresis. Accordingly, the device requires the UV stimulus to 
show memristive switching. A junction between memristive switching and sensing 
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has been reported for a variety of devices [12-14]. To understand the phenomenon 
of stimulus dependent hysteresis and study the potential properties of memsensors, 
a simple three-component model was proposed (Fig. 1c), that allows to investigate 
the interplay between memristive switching and sensing [11]. Within this model, 
in first approximation, the sensor response is assumed to be immediate, while the 
memristive devices can occupy a smooth transition of resistance states between 
a fixed LRS and HRS. As such, stimulus dependent hysteresis at a given voltage 
window was reproduced, as depicted in Fig. Id. In the context of the model, the 
occurrence of stimulus-sensitive hysteresis originates from the difference in potential 
drop over the serial memristive element, that leads to a significantly more pronounced 
hysteresis in the illuminated state. As an additional memsensor feature that goes 
beyond the inherent features of sensors and memristive devices, amplitude adaptation 
to a constant stimulus was modelled (Fig. le), mainly governed by the different rate 
of change in inner state for both memristive devices. This predicts that the junction 
of sensing and memristive switching has the potential to create smart sensor devices, 
that show bio-inspired features like adaptation. 


3 Additive Manufacturing of Sensors 


Self-organized systems are shaped by mechanisms of positive and negative feedback, 
meaning that occurring “positive” events make them more likely to occur again, 
while “negative” events become more unlikely. This feedback-driven dynamic makes 
the system generally robust against fluctuations and will lead it towards its most 
desirable state, defined by the feedback mechanisms. While self-organization is an 
important feature of efficient bio-inspired systems, there are many examples for 
parts of biological systems that are formed by a fixed pattern because they often 
serve a specific purpose e.g. the eye as a means of perceiving visual information. 
Such pre-designed patterns generally make up every technical device, though when 
used in dynamic self-organized systems in an emerging concept like neuromorphic 
computing it is advisable to have fast iteration speed during the development stage, to 
be able to adapt and optimize the design quickly. At the same time as much design- 
freedom as possible should be maintained with respect to the geometry, material 
or composition of parts. In this context the field of additive manufacturing (AM) 
has shown to offer beneficial features for rapid design and production of prototypes 
of both structural and functional components. Traditional manufacturing of parts is 
mostly subtractive e.g. milling, drilling, etc., where material is removed from the 
part until it is shaped as intended. This creates a substantial deal of waste while 
it becomes more difficult and thus expensive the more complex the shape of the 
part is. In contrast to this top-down approach, additive manufacturing (AM) works 
“bottom-up” by assembling the final part out of small fractions. The most common 
way is by stacking thin layers of material on top of each other to create 3D objects 
(see Fig. 2a). For the deposition of material for each layer there are several different 
approaches available, depending on the matrix material and specifications of the 
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(a) Individual layers 


Iterate design 


Fig. 2 Benefits of additive manufacturing: a The bottom-up approach (i.e. stacking layers of mate- 
rial) facilitates more complex geometries, like undercuts in the center of a piece, that would be 
difficult to manufacture by traditional machining. b Quick iteration cycles are made possible by 
directly linking the design and fabrication steps 


part like minimum feature size. Processes like fused deposition modelling (FDM) 
melt a thermoplastic filament and extrude it through a nozzle to deposit the material, 
while stereolithography (SLA) is based on the photopolymerization of thin layers of 
liquid resin. Metal parts can be produced by selective laser sintering (SLS) or the 
conceptually similar selective laser melting (SLM), which heat up a metal powder to 
sinter or melt the grains together [15, 16]. Most commonly object data is provided 
by digital 3D object files, e.g. in the STL-format, created by computer aided design 
(CAD) programs. Object design and fabrication are direct subsequent steps, as seen 
in Fig. 2b. A designer has both large freedom with respect to geometry and materials, 
while also being able to work through a part’s iteration cycles quickly. This led to 
AM also being considered “rapid prototyping” making it a perfect tool to use in 
research and development. 


3.1 Direct Ink Writing of Microparticles 


Direct Ink Writing (DIW) is a subtype of AM, where an ink composed of a carrier fluid 
and colloidal particles with high filler content is extruded onto a substrate through 
a moveable nozzle. The carrier fluid consists of a volatile solvent and possibly a 
polymeric binder. After extrusion the solvent evaporates and the ink solidifies. While 
this technology already has been around for many years in the form of inkjet printers 
for digital images it also got more recent attention as means for AM by depositing 
ceramic particles via the ink [17—19]. DIW allows printing of any materials in powder 
form, particularly materials that are not suitable for other additive manufacturing 
methods or that have complex particle morphologies. This also includes functional 
particles like tetrapodal ZnO (t-ZnO, [20]), which expands the capabilities of AM 
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from purely structural parts towards functional or composite components. However, 
the rheological properties of the ink are crucial for the reliable fabrication with DIW. 
The ink must remain homogeneous and flow evenly to avoid defects or aberrations 
in the print. Colloidal suspension inks are based on a carrier fluid and one or more 
species of suspended micro- and/or nanoparticles. In some cases, a polymeric binder 
has to be added to the solvent increasing the inks viscosity. E.g. if the particles 
are too large to stay suspended in the pure solvent, a polymeric binder is added 
to increase the viscosity preventing the particles from sedimenting. Furthermore, 
complex shaped particles like t-ZnO form highly porous, tangled agglomerates. A 
high viscosity ink is necessary to exert the shear forces required to move the particles, 
otherwise the solvent would flow through the pores separating from them. The carrier 
fluid commonly solidifies either by evaporation of the solvent (e.g. for the systems 
H,O/PEG or EtOH/PVB) or by a chemical reaction (e.g. silicones). Once solid it 
can either persist as a structural matrix for the particles or be removed by thermal 
or chemical decomposition. Especially for functional microparticles it is commonly 
desired to remove the binder to create interfaces to other external species, like gases 
or light, or create percolating or dense structures approximating the properties of the 
bulk material e.g. the electrical conductivity. 


3.2 EtOH/PVB as a Carrier Fluid 


Using ethanol (EtOH) as a solvent and polyvinyl butyrale (PVB) as a polymeric 
binder has shown to yield interesting possibilities, which are further enhancing the 
flexibility of DIW for micro- or nanoparticle based components and can possibly be 
transferred to other similar systems as well. An ECOH/PVB based ink can be printed 
on a glass substrate (see Fig. 3a). Since PVB is a hydrophobic polymer, submerging 
the glass substrate in H,O makes the printed material detach from the glass (see 
Fig. 3b). The result is a flexible free-standing structure, which in itself already opens 
new possibilities towards prints that can be folded to create 2.5-dimensional designs. 
In addition, if slightly wetted with EtOH again, the print can be re-attached to any 
suitable surface, including round or complex ones, on which printing and most other 
deposition techniques (e.g. sputter deposition) are unfeasible or even impossible, dra- 
matically increasing the applicability of printed components (see Fig. 3c). Moreover, 
with a high-power laser instead of traditional unlocalized heating, the heat treatment 
process can be controlled precisely in terms of localization and power transfer. Using 
a 100 W CO; laser two main goals have been achieved: (1) With low power output, 
a gentle milling of a thin portion polymeric binder has been performed on a single 
layer print. Without damaging the embedded microparticles, the top portion of the 
layer has been stripped of its polymeric binder, exposing the functional particles, 
while keeping the bottom portion of the binder as mechanical support Fig. 3d. Even 
after the laser milling process the print could still be detached from the substrate. 
By combining inks of t-ZnO and CNTs an all-printed freestanding sensor device has 
been fabricated with this method, as shown in Fig. 3e. The geometric and mechanical 


272 M.-I. Terasa et al. 


(b) 


(d) CO,-Laser Functional ZnO 
sensor region Functional CNT wiring 


CNTs + 
Polymer Base Polymer Base Polymer Base 


Fig. 3 a Anas-printed layer of an ink of EtOH/PVB with t-ZnO as functional particles. b Submerg- 
ing the glass substrate in water will make the printed layer delaminate. c The delaminated print can 
be re-attached conformably to round or complex surfaces. d Schematic of the gentle laser milling 
process. It allows for precise stripping of thin layers of polymer to expose the functional particles, 
while keeping a base of polymer for structural stability. e A free-standing all-printed UV-sensor 
composed of CNT-ink as wiring and t-ZnO as the sensor component. The print has been laser milled 
to expose the ZnO enabling its sensor functionality. f Separately printed and delaminated CNT- and 
ZnO-prints being assembled freely by slightly wetting the junctions with ethanol 
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flexibility makes such sensor devices a great candidate for components in electronic 
skin applications [21, 22]. Additionally separately printed parts can be assembled 
freely by wetting them at the respective junctions with ethanol (see Fig. 3f) or be 
stored as well as processed mechanically e.g. by cutting, allowing for production 
of stock or modular designs. (2) With higher power output a localized oxidation 
of metal particles has been accomplished, leading to precisely located metal/metal- 
oxide/metal interfaces, which is a promising starting point for simple and accessible, 
yet precise and scalable fabrication processes of MOS sensor devices. 


3.3 High-Viscosity DIW Setup 


The Chair for functional Nanomaterials at Kiel University has established a custom- 
built DIW setup for printing with inks of high viscosities, which is depicted in 
Fig. 4a. It’s extrusion unit is able to exert high pressures up to 40 bar on the ink 
via a steel piston driven by a geared stepper motor. The ink reservoir is a commer- 
cially available polypropylene (PP) syringe encased in an aluminium shell to prevent 
expansion of the PP during extrusion. The extrusion nozzle is a conical syringe tip 
with an orifice diameter of 100 um to 840 um. The nozzle size determines the spatial 
resolution of the print, since a smaller diameter yields thinner lines and thus finer 
features. But at the same time it limits the size, morphology and fill factor of the 
printable microparticles. E.g. tetrapodal ZnO particles (t-ZnO) with an arm length 
of 25 um or more have shown to clog nozzles with 100 um diameter at higher fill 
factors of more than 10 wt% due to the tendency of the tetrapods to become tangled. 
In addition, the print bed is heatable and a 3 W computer controlled blue laser is 
pointed at the printing spot for precise localized heating. Figure 4b shows a selection 
of microparticles, that have been printed with this setup. The setup has been used to 
print all devices presented here and in Sect. 3.4. 


3.4 Gas Sensors by DIW 


Semiconducting metal oxides (MOS) have shown to be a superior material class 
for gas sensing applications, which are essential in the fields of workplace security, 
healthcare or environmental monitoring [23]. The mechanism of gas sensing is com- 
monly attributed to the ad- or desorption of molecules on the solid-gas interface of 
the sensing material changing its electronic landscape, where oxygen is discussed as 
the most popular surface adsorbant species. Oxygen adsorbs to the surface of MOS 
by binding electrons from the conduction band: 


O2 +e% > Oraa) 


In a n-type semiconductor this creates a high resistance electron depletion region 
narrowing the conduction channel, whereas in a p-type semiconductor a hole accumu- 
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Fig.4 a Photograph of the custom-built high-viscosity ink printing setup. b A selection of micropar- 
ticle species that have been printed with the setup, as well as possible applications for each. 


Table 1 Composition of the functional inks for MOS sensor devices 


Sensor CuO/Cu20/Cu CuO:Fe203 

Carrier fluid Deionized H2O Ethanol 

Polymeric binder Polyethylene glycole Polyvinyl butyrale 

Microparticles Copper (15-25 um) Copper (15-25 um) + Iron (40-60 um) 


lation layer (HAL) with lower resistance is created. The sensor response is most com- 
monly discussed in the context of chemical reactions or desorption of surface oxygen 
decreasing either the electron depletion layer or the hole accumulation layer. The ace- 
tone content of human breath has been shown to be significantly increased for indi- 
viduals with diabetic ketoacidosis [24]. Thus acetone breath sensing is a promising 
tool for non-invasive diagnosis and monitoring of diabetic patients, with the aim for 
inexpensive, portable and broadly available sensor devices, giving patients cheaper 
and safer healthcare options, compared to conventional blood tests [25]. Acetone 
gas sensors based on self-organized MOS nanostructures, namely CuO/Cu20/Cu- 
and Fe203/Fe-nanospikes, have been manufactured via DIW and subsequent thermal 
annealing [26, 27]. The composition of the respective inks can be found in Table 1. 

Ink lines have been printed on glass substrates with deposited gold contacts (see 
Fig. 5a—d). The prints have been annealed at 425 °C for 2 h ( CuO/Cu20/Cu) and 
4 h (CuO:Fe203) respectively, which served as means of binder removal as well as 
oxidation of the microparticles. 

During oxidation self-organized spikes of metal oxide have grown outwards from 
the particles, significantly increasing their surface area and generating new hetero- 
junctions between particles, as indicated in the micrographs in Fig. 6a—d. Both the 
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Fig.5 Images of meandering printed ink lines: a Directly after printing. b After deposition of gold 
contacts. ¢ Close-up of a single sensor element. d SEM micrograph of a sensor element. Reprinted 
with permission from [27] 


CuO and the CuO:Fe20; sensors showed great selectivity and sensitivity to acetone 
vapours at an operating temperature of 300 °C or higher (see Fig. 6e+f). 

The proposed sensing mechanism is the reaction of acetone with adsorbed surface 
oxygen: 


CH3COCHs(ads) + 807 (ads) + 8ht —> 3CO, + 3H20 


Since CuO is a p-type semiconductor the sensor response is based on the reduc- 
tion of the HAL. In the case of CuO:Fe203 sensors besides the charge accumula- 
tion/depletion effect, the interfaces between CuO and Fe2O3 nanospikes form p-n- 
junctions that are proposed to increase in resistance by adsorbed acetone providing an 
additional potential barrier. These examples showcased that the technology of DIW 
is perfectly suitable for the design and production of sensing devices. Any species 
and even combinations of microparticles with sensing properties can be deployed 
with great geometric freedom, while allowing one to print on pre-designed circuit 
boards or substrates for immediate integration. 


3.5 Enhancing Sensor Properties by Decoration with Noble 
Metal Nanoparticles 


The gas sensing performance of MOS sensors with respect to sensitivity, stability or 
selectivity can be enhanced by doping as well as decoration with metallic nanopar- 
ticles. To give an example, ZnO as a H, sensor is considered. ZnO nanowires (a 
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Fig. 6 SEM micrographs of the Cu and Fe microparticles after heat treatment. a—d Interface 
between the CuO/Cu20/Cu- and Fe203/FeO/Fe-nanospikes at different magnifications. e Gas 
response of 3D-printed CuO sensor to 100 ppm acetone vapor. f Gas response of CuO:Fe203 
sensor to 100 ppm acetone vapor. Reprinted with permissions from [26, 27] 


n-type semiconductor) are showing H; detection capabilities. A model proposed for 
the Hz sensing mechanism while exposed to ambient air is based on the reaction of 
incoming H) molecules with the adsorbed oxygen to H2O releasing electrons into 
the bulk during this process and thus decreasing the electron depletion region: [28] 


Hy + 40343) > H20 + Se7 


Though, such sensors are influenced as well by ambient parameters like rela- 
tive humidity (RH), so that it is beneficial for the sensor performance to alleviate 
the response to changing conditions [29]. ZnO nanowires that have been decorated 
with Au nanoparticles (AuNP) have shown a significantly reduced susceptibility to 
changes in RH between 30% and 85% [30]. The tentatively proposed sensing mech- 
anism for AuNP decorated ZnO nanowires extends the one for pure ZnO wires by 
taking into account the AuNP/ZnO interface (see Fig. 7). Due to the higher work 
function of the AuNP (®,, = 5.1 eV, ®znq = 4.5 eV) electrons flow from the bulk 
to the interface resulting in Schottky barriers at the Au/ZnO interface thus further 
enlarging the electron depletion region. 

The water desorbs by the heat of the exothermic reaction and by the release of 
electrons the electron depletion layer is reduced. The higher sensor response for Au 
NP decorated ZnO is attributed to the larger depletion layer around the Au NP and the 
induced Schottky barriers, which result in a stronger modulation of the conduction 
channel [31]. The effect of doping and nanoparticle decoration of MOS sensors has 
been studied for a wide variety of material systems. An overview of examples is 
given in Table 2. 
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Fig. 7 Proposed mechanism for the gas-sensing of noble nanoparticle decorated ZnO nanowires. 
a Under ambient air. b Under exposure to H2 gas. Reprinted with permission from [31] 


Table 2 Overview over the effect of doping and nanoparticle decoration on gas sensor properties 
for various material systems 


Base system Nanoparticles Effect References 

ZnO:Fe AgO/Ag Enhanced sensitivity to ehanol vapor | [32] 

ZnO:Ag Ag Enhanced humidity stability [33] 

ZnO:Ag AgAu Enhanced sensitivity towards VOCs | [34] 

ZnO:Ag AgPt Enhanced selectivity for hydrogen [34] 

TiO2 Ag, Au, AgAu, Enhanced selectivity for various gases | [35, 36] 

AgPt 

ZnO:Eu Pd Decrease of working temperature for | [37] 
hydrogen sensing to RT 

ZnO:Pd PdO Enhanced selectivity and sensitivity | [38, 39] 
to hydrogen gas 

SnO2 Zn2Sn04 Enhanced sensitivity to ethanol at RT | [40] 
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Fig. 8 Collection of doped and/or surface functionalized MOS based gas sensors. For each system 
an SEM micrograph as well as a characteristic gas sensor measurement is shown: a+b Fe doped ZnO 
decorated with Ag NP [32]. c+d Ag doped ZnO decorated with Ag NP [33]. e+f SnO, decorated 
with Zn2SnO4 NP [40]. g+h TiO? thin film decorated with AgAu NP [36]. i+j Europium doped 
ZnO decorated with Pd NP [37]. k+l Pd doped ZnO decorated with PdO NP [38]. Reprinted with 
permissions from [32, 33, 36-38, 40] 


Except for the SnO2/Zn2 SnO; system, all nanoparticles have been deposited with 
a Haberland-type gas aggregation cluster source, which is described in more detail in 
the chapter “Memristive Switching: From Individual Nanoparticles Towards Com- 
plex Nanoparticle Networks” and allows for precise tailoring of alloy composition 
during deposition. In Fig. 8 a selection of gas sensors is depicted, with SEM micro- 
graphs showing the morphology and nanoparticle decoration, as well as characteristic 
gas sensor measurements. 

The decoration with nanoparticles has shown to enhance the performance of MOS- 
based sensor devices. A thorough comparison of recent works can be found in [34]. A 
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deeper understanding of the participating interactions and mechanisms will allow for 
a tailored design of specialized or optimized material systems for sensor applications. 
As an example, the concept of the “electronic nose” can be realized by an array of 
highly selective and stable gas sensors, which is discussed as a valuable tool for the 
food industry or environmental applications [41, 42]. 


4 Triggered Assembly 


The growing demand for computational power, arising from the development towards 
AI and the “Internet of Things” [2-4], as well as the resulting energy consumption 
will require highly efficient computer systems, with one of the main drawbacks of 
today’s conventional computers being the losses of the von-Neumann architecture. 
A bio-inspired approach for a computing system based on the mammalian brain 
uses a low frequency but in a highly parallelized architecture in which memory, 
computation logic, energy source and cooling are decentralized. One challenge of 
such a complex system is the communication and thus the interconnections between 
the parallel components analogous to the connections between cortical regions of the 
brain [5]. In a brain interconnections are formed by closely timed spiking of neurons, 
called spike timing-dependent plasticity (STDP), which is, as a Hebbian learning 
rule, commonly summarized as “what fires together wires together”. Neurons that 
spike in a window of a few milliseconds before another neuron nearby, potentiate the 
connection between these improving the efficiency of signal transmission [43]. Ona 
technological level this concept can be transferred to a circuit of initially unconnected 
electronic components. Conductive interconnections between two nearby terminals 
are formed, if both are “active” during a narrow timeframe (but not simultaneously). 
This activity driven formation of conductive interconnections can be established by 
utilizing electrochemical metallization (ECM), where “activity” in this context is 
used to describe the potential of a terminal relative to a reference. The working 
principle of ECM cells is based on applying a voltage to electrodes where at least 
one of them consists of an electrochemically active material (e.g. Cu or Ag), which 
leads to a dissolution of metal ions into an electrolyte between the electrodes. The 
ions drift in the electrical field towards the cathode, where they are reduced and 
crystallize on the surface: 


Anode: M —> M"+ + ne~ 


ļ Drift ļ 
Cathode: M"* +n —> M 


The crystallized metal forms a filament bridging the electrolyte thus reducing the 
resistance between the electrodes as long as the filaments holds. ECM is commonly 
used on the nanoscale as a memristive switching mechanism with an inert counter 
electrode and a solid electrolyte (e.g. SiOz) [44, 45]. However, it is also possible 
to transfer this process to a larger length scale up to a range of several hundreds of 
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Fig. 9 Photographs and current measurement of an electrochemical metallization process over a 
dis-tance of 300 um in an electrolyte of DMSO and MB with copper electrodes under an applied 
voltage of 15 V. The voltage has been applied to the center electrode. The perpendicular adja-cent 
electrodes have been connected to ground. All remaining ones were left floating. a In the beginning 
the current is determined by the conductivity of the electrolyte. b Filaments grow by metallization 
from cathode towards anode. The first filament establishing contact increases the conductance 
significantly. c Further growth leads to thicker filaments, gradually increas-ing the current 


micrometers. Thus, filament growth by electrochemical metallization occurs as well 
for Cu electrodes with a spacing of 300 um, when submerged in a liquid electrolyte 
of dimethyl sulfoxide (DMSO) and methylene blue (MB), which is shown in Fig. 9. 
By applying a voltage of 15 V to the central electrode and connecting ground to 
the adjacent ones, a metal filament has formed from cathode to anode directed by 
the electrical field, significantly increasing the conductance between them. EDX 
measurements revealed, that the filament formed consists of the electrode material, as 
seen in Fig. 10a+b. The liquid state facilitates ion mobility and allows for adjustment 
of electrolyte concentration, which can be used as parameter for the time constants 
of filament growth (see Fig. 10c). 

If a periodic voltage signal is considered for a number of terminals oscillating 
around a resting potential, filament growth acts as positive feedback for closely but 
not exactly synchronized oscillators, since a large voltage drop will occur, where 
repeatedly one terminal is at its maximum while a nearby one is at its minimum. 
Thus, if the voltage magnitudes are properly set and two nearby terminals spike 
with similar (but not equal) frequencies over a sufficient period of time, they are 
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Fig. 10 a+b SEM micrograph and respective EDX map (Cu La! signal) of two copper electrodes 
with grown filament. c Semi-logarithmic plot of time of filament growth over applied voltage for 
two electrolyte concentrations. The slope has been calculated by linear regression 


interconnected by a metal filament drastically increasing the conductance between 
them. The growth time being in the second- to minute-scale acts as a low-pass filter, 
so that only those terminals are interconnected, that show a suitable potential drop 
consistently over a sufficient number of spikes. The larger length scale makes this 
approach a suitable base for integration into small-world setups and furthermore a 
combination with sensor devices allows the terminal activity to be driven by external 
stimuli. 
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5 Conclusion 


The advance towards a data driven society with the “Smart City” and “Internet of 
Things” paradigms requires an ever growing demand of monitoring and sensing 
devices as well as imposing new challenges with respect to performance, energy 
efficiency and data processing but also design, scalability and fabrication of sensor 
devices. In this chapter various concepts have been discussed, tackling these chal- 
lenges. A general model for a memsensor, whose equivalent circuit model consists 
of a stimulus dependent resistor (i.e. sensor) and two memristive components, has 
shown unique features like adaptation to a permanent background stimulus. It serves 
as a foundation for the interpretation of experimental findings and prediction of the 
behaviour of memsensitive devices, facilitating the development of smart, decentral- 
ized sensor systems. The unique features of memsensitive devices can be applied e.g. 
for monitoring the concentration of hazardous gases in a storage cabin. The sensor 
should not react to the slowly changing permanent background which is based on the 
number and kind of chemicals in the cabin, but only to quick changes such as when a 
container is damaged. Furthermore, it has been pointed out that additive manufactur- 
ing is a perfect candidate for the inexpensive, scalable fabrication of sensor devices 
with great design freedom in terms of geometry and material space as well as rapid 
iteration cycles. Microparticle based MOS-sensors have been shown as examples for 
sensor devices fabricated by DIW. An overview over various material system has 
been given, showcasing the enhancement of the performance, selectivity or stability 
of MOS-sensors by doping and decoration with nanoparticles, which allows for tai- 
loring or designing sensor materials for specific applications. Finally, an approach is 
introduced for dynamically establishing conductive interconnections between metal 
electrodes in a liquid matrix. The concept is based on electrochemical metallization 
stimulated by voltage pulses. It acts as a self-organization mechanism to increase 
the conductance between active parts of a circuit resembling STDP. Using sensor 
devices, the “activity” can be driven by external stimuli, which offers a new way of 
introducing neuromorphic plasticity into sensor circuits. 
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Abstract We present an overview of recent developments in the area of acoustic 
sensing that is inspired by biology and realized by micro-electromechanical systems 
(MEMS). To support understanding, an overview of the principles of human hearing 
is presented first. After the review of bio-inspired sensing systems, we continue 
with an outline of an adaptable acoustic MEMS-based sensor that offers adaptable 
sensing properties due to a simple, real-time feedback. The transducer itself is based 
on an active cantilever, which offers the advantage of an integrated deflection sensing 
based on piezoresistive elements and an integrated actuation using thermomechanical 
effects. We use a feedback loop, which is realized via a field-programmable gate 
array or analog circuits, to tune the dynamics of the sensor system. Thereby, the 
transfer characteristics can be switched between active, linear mode, for which the 
sensitivity and minimal detectable sound pressure level can be set by the feedback 
strength (similar to control of the quality factor), and an active nonlinear mode 
with compressive characteristics. The presented sensing system, which is discussed 
both from an experimental and theoretical point of view, offers real-time control for 
adaptation to different environments and application-specific sound detection with 
either linear or nonlinear characteristics. 
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1 Introduction 


Current technological sound and speech processing systems typically consist mainly 
of three parts (schematically shown in Fig.1): a sensing unit, a pre-processing 
unit and the sound analysis/processing unit. The sensing unit consists of one or 
more (typically capacitive) microphone based on micro-electro-mechanical system 
(MEMS)-technology with linear transfer characteristics and 120dBA as maximal 
detectable sound pressure level (SPL), a self-noise floor (corresponding to the mini- 
mal detectable SPL) of 20 — 30dB (absolute minimum at the moment (non-MEMS): 
x~ 7dB), and a frequency range of 20Hz-20kHz [1]. The pre-processing unit is 
applied to tune the sensed signals for easier processing/analysis. It typically includes 
a pre-amplifier, amplifying the microphones signals, and filter stages, decomposing 
the signal into different frequency bands, and additional stages to include e.g. weight- 
ing functions, correcting the signal and/or digitalizing it. The processing unit is based 
on signal processing stages and/or on learning systems such as convolutional, recur- 
rent and spiking neural networks. Thereby, the pre-processing of the signals shall 
improve the processing performance by extracting important sound features, which 
are then fed into the neural networks (see e.g. [2]). Typical low-complexity features 
are, for example, the envelope of the sound signal, the RMS energy, the frequency 
(given by the zero crossing rate or the spectral centroid) or spectral spread or flux. 
Alternatively, the sound data is fed directly into the NN, known as end-to-end learn- 
ing, or first transformed into a time-frequency representation, known as spectrogram, 
before it is fed into the network. The goal is to separate the relevant information like 
speech from irrelevant one like noise. 

The performance of speech processing systems has strongly increased in recent 
years [3-7]. This can be mainly attributed to the improvements in the pre-processing 
stage and the sound processing networks, since microphone technology including 
readout is well developed with easy and cheap fabrication, small size and large 
dynamic ranges and performance. The pre-processing stage typically includes pre- 
amplifiers, amplifying the microphone signals, and filter stages, decomposing the 
signal into different frequency bands, and additional stages to include, e.g., weight- 
ing functions, correct and/or digitalize the signal. The development of pre-processing 
stage was driven mainly by implementing bio-inspired pre-processing (e.g. nonlin- 
ear amplification) as well as developing extensive noise cancellation algorithms. 
Thereby, it was recently shown that nonlinear, bio-inspired pre-processing based on 
frequency filtering and nonlinear amplification of signals is particularly important 
for sound processing, since it yields strongly increased success rates for speech clas- 
sification tasks [8]. The increasing demands in computation power and energy can be 
addressed by neuromorphic, bio-inspired pre-processing implementations in hard- 
ware like silicon cochlea, Hopkins electronic ear, AER-EAR, or the FPGA cochlea 
[9]. These model sound processing by the cochlea in the inner ear, as schematically 
shown in Fig. 1 and, compared to software implementations, offer the advantage 
of low latency, real-time performance, reduction of redundant information and data 
streaming (due to asynchronous event-based spiking output) and lower power con- 
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Fig. 1 Comparison of steps in technological speech processing systems and the human hearing 
system. Detailed description of human hearing is given in chapter 2. BM-basilar membrane, OHC- 
outer hair cell, IHC-inner hair cell, AN-auditor nerve 


sumption. Noise cancellation algorithms can be applied to improve sound process- 
ing in more complex hearing situations with multiple sources, but these increase the 
necessary computing power and energy consumption. In both cases, the idea is to 
improve the sound feature representation in the audio signals to allow a better sound 
processing in the neural networks. 

Despite these developments, room reverberation (echos), interfering noise, or any 
other perturbation to the signal can critically affect the underlying feature represen- 
tation and thus limit sound processing performance, in particular, at low signal-to- 
noise ratios (SNRs) [10-12]. Further challenges arise from the inability to separate 
individual sound sources from a mixed acoustic signal, to generalize to unknown 
acoustic conditions, and to run on low-power embedded devices. Adaptation of the 
system parameters can help to overcome the issues arising from changing acoustic 
environments such as room reverberations, noise level etc. Current research focused 
on adaptation of the pre-processing stage: changing the gain of the pre-amplifier 
[13], the settings of the filter banks [14], how signals are combined for improved 
directionality [15], microphone switching to increase dynamic range [16] or auto- 
matic gain control in neuromorphic systems [17, 18]. Although methods to adapt the 
sensing properties of microphones have been published, e.g., adjusting sensitivity by 
changing the bias voltage or effective microphone area [19], these are not (widely) 
applied. While the above described developments could improve sound process- 
ing performance for standard hearing conditions (combined with higher computing 
power demands), until now satisfying solutions for reasonable sound processing of 
signals in noisy environments, i.e., with low signal-to-noise ratios, low volume of 
important sounds or loud masking sounds, could not be derived. 
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Thus, improving the sensor stage is the next logical step to bridge this gap. In this 
chapter we will review (some of) the research in bio-inspired acoustic sensors. For 
a better understanding, we will shortly summarize what is known about the acoustic 
sensing in humans until the transduction stage, including the processing and encod- 
ing mechanisms occurring at this level. Following this, several bio-inspired sensors 
will be described in relation to their processing capabilities, in particular frequency 
decomposition and nonlinear filtering of the sound signals, and their adaptation 
properties, which are the most important functionalities in acoustic bio-inspired sens- 
ing. Finally, we will present a bio-inspired acoustic sensor [20-22] that includes all 
the above mentioned three functionalities, i.e. frequency decomposition, nonlinear 
amplification and adaptation. 


2 Human Hearing/Auditory Pathway 


In this section, we will give a short overview of the hearing process until the trans- 
duction of acoustic signals with a particular focus on the pre-processing of the signal 
taking place before the actual sound analysis in the cortex (see Fig. 1). The descrip- 
tion is mainly based on the information given in [23, 24] and is separated according 
to the three parts of the ear (see Fig. 2): (i) the outer ear, which includes the pinna and 
the external auditory canal, (ii) the middle ear with the ossicular chain consisting of 
incus, malleus and stapes, and (iii) the inner ear with the cochlea. 


Outer ear Middle ear Inner ear 


Semicircular 


Malleus Canals 


Cochlear 
nerve (VIII) 


Cochlea 


External 
auditory 
canal 


Pinna 
Tympanic 16 kHz 


membrane Stapes 
(attached to 
oval window) 


Fig. 2 Schematic illustration of the ear. Modified from [25] 
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Human hearing starts with the pinna, which funnels the sound signals into the ear 
canal, further transferring the signal to the tympanic membrane. Even in this very 
first step of the hearing process, several steps before the actual transduction of sound 
signals into electric signals (action potentials/spikes), a pre-processing/conditioning 
of the acoustic signals is present, which shall support later sound processing and 
sound perception. In detail, the transfer characteristics of the outer ear pinna and ear 
canal is on the one hand direction-dependent, providing localization cues, and on the 
other hand includes frequency-selective amplification of sound signals. The latter 
amplifies the sound signals in the important range of 200Hz to 10kHz, with particu- 
larly strong amplification in the range of 2-4kHz, the range of highest sensitivity for 
hearing [26]. This pre-conditioning is mainly constant and not adaptive. 

In the next step of the hearing process, the vibrations of the tympanic membrane 
due to sound excitation are transferred to the inner ear by the ossicular chain in the 
middle ear. Its task is to provide an impedance matching between the air-based sound 
in the outer ear and the fluid-based sound propagation in the inner ear. Without this 
matching, most of the sound energy would be reflected at the air-fluid interface whith 
areflectance of approximately 99.9%. The transfer characteristics is almost flat in the 
range between 200Hz and 8kHz (small peaks around 1kHz-2kHz and 4-5kHz) and 
decreases outside this range. Important to note here is that the efficiency of sound 
transfer can be adapted for small frequencies below 2kHz by contraction of the 
stapedius muscle. Its contraction yields an increase of the stiffness of the ossicular 
chain , thus reducing the sound energy transferred to the inner ear. The main purpose 
of this adaptation is a protection of the inner ear from damage due to loud noises and 
an attenuation of the perception of ones own voice. 

The final part in the acoustic sensing process is the inner ear, the cochlea to be 
accurate. The cochlea itself, a bony, snail shell-resembling structure consists of three 
fluid-filled chambers: the scala vestibuli, the scala tympani and between both the 
scala media, hosting the organ of corti (see Fig.3). Thereby, the scala vestibuli is 
separated from the scala media by a thin membrane, the Reissner’s membrane. On the 
lower side, scale media is separated from the scale tympani by the basilar membrane 
(BM). 

The pressure wave, initiated by the stapes at the oval window of the cochlea, 
propagates through the scala tympani up to the apex of the cochlea, where it is trans- 
ferred to the scale vestibuli through a small connection of both chambers and then 
propagates in this chamber down to the base of the cochlea until finally reaching the 
round window (see Fig. 4). In response to the propagating sound wave, the basilar 
membrane and Reissners membrane oscillate. Thereby, due to a varying stiffness 
and thickness of the basilar membrane along the cochlea length, resonance occurs 
at different locations in the cochlea depending on the frequency of the sound wave 
(traveling wave theory), yielding an effective frequency decomposition of the input 
signal. This encoding of sound frequency into a location/place along the cochlea is 
termed tonotopy and the frequency-place coding is kept throughout the complete 
auditory pathway up to the auditory cortex in the brain. Thereby, humans can dis- 
tinguish frequencies differing only by 0.1% [28]. The sharpness of the frequency 
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stria 
vascularis 


Fig. 3| Schematic showing the cross-section of the cochlea with focus on the organ-of-corti, which 
is the place of sound transduction. Thereby, the organ of corti is located in the the scala media, one 
of the three fluid-filler chambers of the cochlea with scala tympani below and scala vestibuli above. 
The three chambers are separated by Reissner’s membrane on top and basilar membrane below, 
which hosts the organ of corti. From [27] 


resolution and cochlear sensitivity is discussed to be further influenced by the lon- 
gitudinal variation in stiffness of the tectorial membrane (TM) [29]. 

Situated on the basilar membrane and connected on top with the tectorial mem- 
brane in the scala media is the organ of corti (OoC), which hosts the actual acoustic 
transducers/receptors of human hearing, namely the hair cells. These are cells with 
cilia on top, giving them their name. In the inner ear, there are two types of hair cells, 
the inner hair cells (IHC) and the outer hair cells (OHC), with different functionali- 
ties, morphology and purpose [23, 31]. Both of them have three rows of differently 
sized cilia on top, whereby these are V or W-shaped on top of outer hair cells and in 
a straight line for inner hair cells. 

The inner hair cells perform the acoustic sensing/transduction in the following 
way. The sound pressure wave deflects the cilia on top of the inner hair cell. If 
the cilia are bent towards the largest ones, potassium channels on top of the cell 
open. The potassium influx thus polarizes the hair cell and triggers the opening of 
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Fig. 4 Schematic of a traveling wave propagation in the cochlea. From [30] 


calcium channels at the bottom. This in turn initiates the release of neurotransmitter 
glutamate into the synapse between hair cell and acoustic nerve. Glutamate then 
activates the connected acoustic nerve, yielding to an increase of its spike rate. If 
cilia are unbend/not deflected, a part of the potassium channels is open, which yields 
a constant small neurotransmitter release and a constant spiking of the acoustic nerve 
cell. If the cilia are bent into the other direction, i.e., towards the smallest cilia, (during 
the second half of the sound sine wave) potassium channels close, calcium channels 
close and no neurotransmitter is released. This yields a reduction of spike rate at the 
acoustic nerve. Thus, the sound pressure changes are transformed into the changes 
of the spike rate at the acoustic nerve (electrical signal). 

Here, additional encoding of sound information takes place, in particular of sound 
amplitude, sound frequency and time course of sound signals, as summarized in 
Table 1. 

The above described phase locking of polarization/hyperpolarisation of the hair 
cell with the sound input is possible only up to frequencies in the range of 2kHz. The 
phase locking is another encoding of sound frequency, additional to the frequency- 
place code discussed above. For frequencies larger than 2kHz, the alternating compo- 
nent of the hair cell potential decreases while the DC component, also termed direct 
component, increases due to incomplete repolarisation of the cell. In response to this, 
the spike rate on the auditory nerve will increase strongly at first, but then decrease 
fast and settle at a certain plateau value in response to a constant pure tone input (see 
Fig. 5). This is termed rapid and short-term adaptation and can be modelled using 
e.g. exponential and power-law adaptation [32]. At the offset of the sound input, the 
spiking rate is strongly reduced below the spontaneous spiking rate (even up to ces- 
sation) and slowly recovers to the spontaneous spiking rate. Besides the duration of 
the stimuli, the adaptation behaviour depends on previous stimuli, sound amplitude, 
previous stimulation history, and the spontaneous rate of the fibre [33, 34], making 
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Table 1 Pre-processing in the sensing process. Abbreviations: BM-basilar membrane, IHC-inner 


hair cell, OHC-outer hair cell, AN-auditory nerve 


Location 


Pre-processing 
functionality 


Functional relevance 


Realisation of 
functionality 


Outer ear 
pinna and ear canal 


Direction-dependent 
transfer characteristics 


Localisation 


Shape of pinna 


Frequency-selective 
amplification 


Amplification of 
important frequency 
range 


Shape of ear canal 


Middle ear 
ossicular chain 


Damping of sounds 


Protection of inner ear 
from loud sounds 
(incl. own voice) and 
reducing masking by 
own voice 


Contraction of 
muscles reduces 
flexibility of ossicular 
chain/reduced 
efficiency in transfer 


property 


Inner ear Frequency Sound analysis Resonance behaviour 

BM, IHC decomposition of BM (varying 
stiffness and mass), 
phase locking of IHC, 
filtering properties of 
AN 

OHC Compressive Detection of sounds Active feedback from 

amplification near or below thermal | OHC (by somatic 


(nonlinear filtering) 


noise levels, dynamic 


motility and hair 


range increase, bundle motility) 
amplitude-dependent 
resolution 
IHC-AN synapse, Adaptation Improve perception in | Efferent feedback, 
OHC noisy environments, | which can damp OHC 
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Fig. 5 Short-term 1000 ------- pasean r-r weeeeegeseeeey 
adaptation of spike rate ' : : : ' ' 
observed at synapse between 
inner hair cell and auditory 
nerve for a constant 
amplitude sound input 
lasting 500ms (from [32]). 
Shown is the histogram 
obtained from using the 
model by Zilany et al. [32] 
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it rather complex for modeling. The mechanism underlying the adaptation process is 
not completely understood. Discussed reasons are the depletion of neurotransmitter 
or the desensitization of post-synaptic receptors (see e.g. [35, 36]). Whether this 
adaptation is important for the sound processing is unclear. A discussed functional 
relevance of this adaptation is the shaping of the onset response, highlighting the 
sound onset and offset, which might be important, e.g., for time-of-arrival detection 
used in localisation. 

Furthermore, the amplitude of the sound is encoded in the spike rate in three 
different ways. First, for a single fibre of the auditory nerve, the spike rate depends 
nonlinear (sigmoidal) on the amplitude of the receptor (hair cell) potential, i.e., 
for potentials below and above a certain threshold the spiking rate is constant and in 
between these thresholds a correlation between spike rate and amplitude is observed. 
The range between thresholds is small compared to the dynamic range of hearing, 
but enables a high resolution of amplitude encoding. To cover a larger dynamic 
range, multiple fibres with different sensitivities (from low over medium to high) 
are connected to one IHC. The combination of their spike rates in response to a 
sound enables the encoding of a larger amplitude range. For low volume sounds, 
only the highly sensitive fibres are encoding the sound amplitude. For larger sound 
levels, the high sensitivity fibres enter the saturation regime (higher threshold), while 
the medium fibres cross their lower threshold, thus encoding the amplitude of the 
stimulus. For even higher sound levels, medium sensitivity fibres saturate and low 
sensitivity fibres can encode the amplitude. For very large sound levels, the region 
of activated sound fibres becomes much larger then expected from the characteristic 
frequency. In this case, also the number of activated fibres (size of activated area) 
encodes the sound amplitude. This combination of different sensitivities helps to 
encode a large dynamic range. 

Nevertheless, the hair cell has to respond to a large range of amplitudes (120 dB) 
and sound pressure levels (SPL). For the lower SPL, only sub-nm deflections of 
cilia are generated and the energy of the sound wave exhibits energy levels in the 
range of thermal noise [37, 38]. Reliable sensing of low SPLs and the large dynamic 
range would not be possible for purely linear sensors. Here another effect enables 
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the large amplitude range, namely a compressive (nonlinear) amplification before 
the transduction. Thereby, two mechanisms are discussed as main source for the 
nonlinear amplification, which are hair bundle motility and somatic motility of outer 
hair cells [39]. The hair bundle is the group of cilia on top of a hair cell. These 
are linked by filaments, called tip-links. If these tip-links are stretched, potassium 
channels open (starting the activation of the hair cell+ auditory nerve). An adaptation 
mechanism was proposed, which yields a downward movement of the plate of the tip 
link and thus a reduction in its stretching upon a constant force. This creates a region of 
negative stiffness in the otherwise linear force-displacement relationship [24]. Due to 
the nonlinear relationship, the hair bundle dynamics are nonlinear as well, including 
nonlinear dynamics effects like bifurcation to self-excited oscillations etc. The second 
discussed effect is somatic motility [40], which is a stretching and compressing 
movement of the outer hair cell, changing its length. For reviews, see [31, 39, 41]. 
Here, the influx of potassium (after deflection-induced channel opening) results in 
the dislocation of chloride ions from prestin molecule. Prestin is a molecule abundant 
in the OHC membrane, which changes its shape depending on the voltage due to the 
insertion or removal of chloride ions. If chloride ions leave the prestin molecule, it 
compresses, which yields a compression of the outer hair cell. This happens when 
the basilar membrane moves upwards (towards the tectorial membrane). Since the 
outer hair cell is connected to both (TM and BM), the compression of the OHC yields 
an amplification of the upward BM motion, i.e. an amplification of the sound input. 
For the other case, i.e., BM moves downwards (away from TM), potassium channels 
of OHC close, resulting in the returning of chloride ions into the prestin molecule, 
which in turn stretches. Thus, the OHC stretches and amplifies the downward motion 
of the BM. 

Besides mammalians, only birds, crocodiles, frogs and lizards have two types 
of hair cells, whereby only for mammals one might be capable of somatic motility 
[40, 42, 43]. Nevertheless, amphibians and reptiles like frogs, turtles, and lizards 
exhibit nonlinear dynamics in the hearing process as well [44-49]. Therefore, it 
was proposed [24, 43, 50, 51], that the hair bundle motility is the reason for the 
nonlinear amplification but has a low gain of a factor of ten, while somatic motility 
is intrinsically linear but offers an additional amplification (driven by hair bundle 
motility) to reach the amplification factor of 1000 observed in mammals. 

Besides the frequency decomposition, nonlinear amplification and short-term 
adaptation described above, further pre-conditioning of the signals occurs in the 
inner ear, driven by an efferent feedback from subsequent processing stages [52- 
55]. This feedback is proposed to control OHC motility by a calcium driven motility. 
It is expected to be used to reduce unwanted/unattended (non-relevant) contributions 
in the sound signal by damping the amplification of OHCs. The efferent feedback 
controls thereby a larger group of OHCs simultaneously (as seen from innervation), 
but can address specific frequency ranges individually. The efferent feedback was 
shown to improve speech perception in particular in noisy environments. It is dis- 
cussed that this improved perception is achieved by the active damping (or the lack of 
active amplification) for the noisy frequency bands in comparison to the amplification 
of the frequency bands associated with speech (or important sounds). 
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Concluding, in the hearing process up to transduction of the signal into electrical 
signals, a number of pre-processing steps are involved, which are thought to improve 
sound/speech perception and increase efficiency of the system. The most impor- 
tant pre-processing steps thereby are the frequency decomposition of the signal, the 
nonlinear (compressive) amplification and the various adaptation mechanisms. 


3 Bio-inspired Acoustic Sensing 


Does it provide advantages if bio-inspired pre-processing, as discussed above, is 
introduced into technological speech processing systems? Indeed, several researchers 
could demonstrate that integrating bio-inspired sensing (including a pre-processing) 
can strongly improve the performance of sound/speech processing systems. The first 
example is given by Araujo et al. [8], who demonstrated recently that the nonlinear 
filtering is an important factor for successful speech recognition. They used the task 
of spoken digit recognition to analyse the effect of the bio-inspired pre-processing 
onto the word success rate of their sound processing system. Therefore, they applied 
a frequency filtering to sound samples recorded with a microphone, and added vari- 
ous nonlinear amplification/filtering methods up to a cochlea-like nonlinear filtering. 
Then, the pre-processed signals were fed into a neural network for solving the recog- 
nition task. Araujo et al. [8] could show that the neural network itself had a success 
rate of 70-80% if combined with linear frequency filtering, whereas the combination 
of nonlinear filtering and neural network reached up to 96% recognition rate. They 
could determine that the contribution of the nonlinear filtering to this recognition rate 
was nearly 80%. Even for noise conditions (sound samples from subway, car etc.), 
high recognition rate of up to 86% were achieved, if nonlinear filtering was applied. 

Another example for the advantage of bio-inspired sensing over conventional 
systems was given recently by Wang et al. [56]. They developed a sensor based on 
resonant operation, in contrast to the below resonance operation of microphones, 
and tested its effect on machine learning-based biometric authentication (speaker 
identification). It was shown that an exceptional error rate reduction in speaker iden- 
tification with their bio-inspired sensor in comparison to MEMS microphone based 
system is achieved using only a small amount of training data. 

The third example is the work of Kiselev et al. [18], who could demonstrate 
in their work the importance of adaptation in acoustic systems. They integrated an 
automatic gain control in their dynamic acoustic sensor, which changes an attenuation 
before and/or an amplification after the band-pass filtering, to keep the spike rate of 
their system in a pre-defined range. The application of this adapting sensing (pre- 
processing) system for differentiating speech from noise showed that the system 
performed much better for low signal-to-noise ratios (SNR), i.e. up to 15% increase in 
accuracy than the system without adaptation. Speech or sound in low SNR conditions 
is an acoustic environment with which speech processing systems still struggle. 

Since it seems to be advantageous to integrate the bio-inspired pre-processing, 
we will discuss in this chapter several bio-inspired acoustic sensing systems, which 
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are capable of frequency decomposition, nonlinear amplification and/or adaptation. 
Sound/speech processing systems, incorporating or adapting mechanisms from the 
biological hearing, can be divided into two groups: (i) sensors with pre-processing 
properties (bio-inspired sensors) and (ii) systems incorporating models of pre- 
processing in the processing circuits. Before we will describe the bio-inspired sensors 
with integrated pre-processing in more detail, we will give a short introduction to 
systems with pre-processing in the stages after transduction in the next section. 


3.1 Systems with Bio-inspired Pre-processing After the 
Sensor 


Systems that incorporate pre-processing in circuits after the microphone are the stan- 
dard case for speech processing systems like smartspeakers or in hearing aids etc. 
These apply mostly digital signal processing to reduce noise in signals, separate dif- 
ferent speech signals from each other or amplify certain frequency bands. Despite 
the advantages due to the development of these software-based implementations, 
these have several drawbacks in terms of computation power, power consumption, 
latency and data-streaming and are typically not located near the sensor node but 
rather cloud-based. Here, hardware-based implementations can improve the perfor- 
mance in terms of the drawbacks listed above enabling a local and real-time per- 
formance. Neuromorphic cochleas have been designed starting from the late 1980s 
[57] and ongoing development resulted in a 64 channel binaural audition sensor, 
which is capable of speaker identification, source localization and was shown to be 
computationally less demanding [58]. Examples of neuromorphic sound processing 
platforms are the so-called silicon cochlea [57, 59], the Hopkins electronic ear [60], 
AER-EAR (also named dynamic auditory sensor) [9, 61], or the FPGA cochlea [62, 
63]. These consist mainly of (i) a number of cascade or parallel filter banks, (ii) 
followed by nonlinear amplification stage and finally (iii) a spike generation stage. 
Furthermore, active coupling between filter stages [64] was introduced to improve 
roll-off of frequency response as well as the synchronization between two silicon 
cochleas for source localization [65]. Different adaptation mechanisms are succes- 
sively included. Automatic gain control of the OHC model stage is included as a 
model describing feedback from medial olivocochlear nuclei [66]. 


3.2 Bio-inspired (Acoustic) Sensors 


This section focuses on bio-inspired sensors, which integrate bio-inspired pre- 
processing in the sensor/sensor properties itself rather than adding it as an addi- 
tional stage after the sensing stage. Thereby, we will review the different systems 
according to the integrated pre-processing, namely frequency decomposition, non- 
linear/compressive amplification and adaptation. 
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Frequency Decomposition 


To incorporate frequency decomposition into the sensing system, resonant opera- 
tion is applied. In contrast to this, microphones typically operate below resonance 
to guarantee a linear transfer characteristic over a large frequency range. Resonant 
operation offers two advantages, namely a higher sensitivity than non-resonant oper- 
ation and a band-pass filtering functionality. Thereby, the increase in amplitude at 
resonance compared to out-of-resonance mode and the bandwidth of the filter are 
closely related and determined by the damping in the system. Quantitatively, the 
damping can be described the quality factor of the system. A higher quality factor 
indicates a lower damping, which results in larger amplitudes and a smaller band- 
width compared to lower quality factors. The resonance frequency in bio-inspired 
acoustic sensing systems is mainly determined by the geometrical dimensions of 
the sensor. Two geometric approaches are mainly applied: either beam structures or 
membranes, and examples for both are shown in Fig. 6. 

Beam structures, sometimes termed artifical hair cells (AHC), can be either 
mounted single-sided or double-sided. To tune the frequency, typically the length 
or width of the beams is varied. The coverage of a larger frequency range is obtained 
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Fig. 6 Examples of a beam-based and b membrane based bio-inspired acoustic sensors (from [56, 
68, 71] 
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using multiple beams with different resonance frequencies. However, a system of 
beams covering the complete auditory range was not yet realized, possibly due the 
small bandwidth and the thus large number of beams necessary. The read-out of the 
sensing signal is realized in various ways: piezo-electric, capacitive, piezo-resistive 
or optical. 

Membrane structures, also termed artificial basilar membranes (ABM), typically 
have a triangular shape to model the frequency decomposition of the basilar mem- 
brane. In the case of optical readout, the analog coding of the frequency can be kept. 
In most cases, however, additional electrodes are added for easier readout/simpler 
calibration, which convert the membrane deflection into electrical signals. These are 
often based on the piezoelectric effect [56, 67, 68], but can be piezo-resistive or 
capacitive as well. 

Operation was demonstrated in air or fluid for beam as well as membrane-based 
devices. Some of these devices were already tested for implantation to activate the 
inner ear [67, 68] in guinea pigs. Here, a brainstem response could be observed, 
demonstrating the activation of the auditory nerve by the implant. 

Besides membrane and beam-based structures, also graded material properties 
using, e.g., acoustic meta-materials are applied for frequency-space coding [69, 70]. 


Nonlinear Dynamics 


Diverse methods exist to tune sensors, not only acoustic ones, into a nonlinear regime 
to improve sensing properties like dynamic range, bandwidth, and (nonlinear) ampli- 
fication and to exploit ensemble effects. Among these are feedback or feedforward 
loops [72-76], hydrodynamical coupling [77], electrostatic interaction [78], elastic 
properties, multi-mode coupling, and coupling of resonators [79]. 

Thereby, some of the feedback-based systems try to explicitly model the biological 
processes like hair bundle adaptation or somatic motility [73-76]. Unfortunately, 
their application to sound was either not yet achieved due to the size of the sensor 
[73, 74] or exhibited only a constant output level. The feedforward system by Crowley 
et al. requires a priori-knowledge on the input stimuli [72]. 

Besides the improvements for the sensing (like dynamic range, resolution etc), 
nonlinear, mechanical resonators offer the possibility to bring computation to the 
sensor domain, as was recently shown [80]. Thereby, single or networks of nonlinear, 
mechanical oscillators can be used to implement reservoir computing [81, 82]. The 
nonlinearity of the oscillators was applied furthermore to implement bit storage and 
bit flip operations [83, 84] and coupling of several mechanical oscillators can be used 
to realize logic circuits like binary comparators and XOR/NAND logic gates [85]. 


Adaptation 


First attempts to realize the idea of adapting the sensor based on the computation of 
the sensed signal were undertaken only recently [86, 87]. Tsuji et al. [86] developed 
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Fig. 7 Improved adaptation principle by Yamazaki et al. [88] 


an artificial cochlea membrane, which resonates at different positions depending 
on the frequency. They used a two-step process for adaptation. In the first step the 
resonant position was detected and in the second step (control step) the oscillation 
at the neighboring positions was damped by applying a feedback signal. Further 
development [88] improved the speed of adaptation (Fig. 7). 

Guerreiro et al. [87] used a single, mechanical resonator and fed its sensing signal 
to a leaky-integrate and fire neuron model, implemented in a microcontroller unit. 
The resulting pulses are used to apply an AC signal changing the Q-factor of the 
system to model short-term adaptation and a DC signal using a charge-pump circuit 
for changing the spring constant of the system to model long-term adaptation. 


4 Recently Developed Adaptive, Acoustic Cantilever Sensor 


The acoustic sensor, which we developed [20-22, 89], is based on a beam structure 
with appropriate feedback (see Fig. 8). Thereby, the beam is a multi-layer structure, 
build from silicon, silicon dioxide and aluminium with a typical length of 350 um, 
a width of 150 um and a thickness in the range of 1 — 5um. For sensing of the 
deflection, piezoresistive elements are integrated near the base of the beam. A thermo- 
mechanical actuation principle is integrated by the aluminium loop on top, which is 
heated upon voltage application. Due to the different thermal expansion coefficients 
of the different materials, the beam bends upon heating, i.e., due to the voltage signal 
applied at the aluminium loop. The beams, sometimes termed active cantilevers, were 
originally developed for application in atomic force microscopy [90, 91], but later 


302 C. Lenk et al. 


___MEMS cantilever __ Piezoresistive readout & signal __ 
Sound Foyt | Pre-amplification u, { 


[Deflection x F 


| High-pass Sensor signal 


Feedback signal tac 


Thermal actuation ô [> 


Fig. 8 Top: Schematic of the sensing system including the cantilever, feedback, and thermal actua- 
tor. All variables as in Eqs. (4)-(6). Bottom: Microscope image of the MEMS cantilever, consisting 
of silicon body, Al actuation loop (red) and piezo-resistive deflection sensing (green) 


on successfully applied in scanning probe lithography, gas flow sensing and particle 
detection, IR sensing and other fields. Details on beam fabrication and applications 
can be found in several reviews. See, for instance, Refs. [92, 93]. 

The feedback loop, used to tune the dynamics of the beam, consists of (i) reading 
the sensor signal, (ii) calculating the feedback function and (iii) using the feedback 
signal to drive the actuator (see Fig. 8). The calculation of the feedback signal is 
done in an FPGA architecture on a STEMlab 125-14 board, which allows a near 
real-time feedback. Before the feedback is calculated, the sensor signal is amplified 
by a factor of 1000, high-pass filtered to use only the AC signal, and finally digitized 
by an analog-to-digital converter on the STEMlab-board (sample rate 125 MHz and 
14bit resolution). While the input range of the STEMlab board can be switched 
between 1 V and 20V, the output range is limited to 1 V. Values of the feedback 
signal, outside of this range are mapped to the maximal value. After the calculation, 
the feedback signal is converted into an analog voltage signal by the digital-to-analog 
converter of the STEMlab board (sample rate 125 MHz), fed to a buffer board and 
finally applied to the actuator of the beam. 


4.1 Frequency Decomposition 


The intrinsic frequency wọ = 22 f of the cantilever sensor is determined by the length 
ls; and the thickness ds; of the sensor together with the elasticity module Es; and 
density psi for Si as follows: 
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wo dsi Esi 
f= = 8 i (1) 
20 2xlg; 12psi 


where ô, denotes the pre-factor for the n-th mode. Here, the first transverse mode is 
considered only. The material parameters of the cantilever are given in Table 2. The 
considered sensor dimensions yield a frequency of the beam of f ~ 14 kHz. 

From the device parameter, we can calculate a quality factor Qo of the cantilever 
by means of the formula for an oscillating beam in air that was derived by Zoellner 
et al. [94]. It is mainly determined by the damping due to the surrounding fluid and 
given by 


Apsidsi 3.7997 
—PSES + 1,05333 + 
T WSi Pgas J/2Re 
Qo = 3.8019 , 2.7364 (2) 


V2Re a 2Re 


with the Reynolds number Re for this system given by 


2 as 2 
Re = 2 f Peas WS; (3) 
Angas 


Peas and Ngas denote the density and the dynamic viscosity of the surrounding media, 
i.e., air, respectively (cf. Table 2). For the investigated sensor, we use a quality factor 
of Qo ~ 43.2 in simulations (Q ~ 40 — 50 in experiments). 

However, upon adding a feedback we see a decrease of bandwidth of the sensor 
(see Fig.9). This effect can be described by a introducing an effective quality factor, 
which depends on the feedback strength. As was shown recently [22], the effective 
quality factor not only describes the change in bandwidth of the system but also the 
change in sensitivity (for the linear regime). 


Table 2 Parameters of the sensor system and surrounding fluid (air) 


Parameter Value 
Width of beam wsi 150 pm 
Length of beam lsi 350 pm 
Thickness of beam dsi 1.25 ym 
Density Silicon pg; 2329 kg/m? 
Young’s modulus Silicon Es; 170 GPa 
Mode factor 5; for frequency calculation 1.8751 
Density air Pgas 1.189 kg/m? 


Dynamic viscosity air ngas 18.232 - 1076 Pas 
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Fig. 9 Frequency response of bio-inspired sensor for different feedback strengths and constant 
sound pressure level 


4.2 Nonlinear Dynamics 


Human hearing system demonstrates nonlinear transfer characteristics, as was 
described in Sect.2. In Fig. 10, the response of our developed sensor is shown in 
dependence of the sound pressure level for different values of the feedback strength 
a obtained from measurements. If the feedback strength is less than the critical 
feedback strength, linear transfer characteristics are observed. Thereby, the sensitiv- 
ity strongly depends on the feedback strength (effective Q-factor). Near the critical 
feedback strength the transfer characteristics become nonlinear, whereas for larger 
feedback strengths autonomous oscillations are observed even without sound input. 
Thus, by tuning the feedback strength, nonlinear characteristics, comparable to the 
biological ones, are obtained. 

To understand this transition from linear to nonlinear dynamics, we perform non- 
linear dynamic analysis and simulations of the system, which are presented in the 
following. The modelling of the cantilever sensor is an extended version of the modal 
description proposed by Roeser et al. [96]: 
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Fig. 10 Experimentally determined amplitude of oscillations in dependence of a for different sound 
pressure levels. From [95] 
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In Eq. (4), the dynamic variables x, 0, and u ac denote the deflection of the cantilever 
(at the free end), the temperature difference between the beam structure and the 
surrounding, and the high-pass filtered version of the sensing voltage us, respectively. 
The latter is linearly related to the deflection with the scaling factor k: 


Us(t) = kx(t). (5) 
The actuation voltage uact(t) is given by 
Uact(t) = auac(t) + upc (6) 


with a feedback strength a and a bias voltage upc. The model parameters in Eqs. (4) 
include the transfer factor œ from temperature into deflection, the time constant 
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Table 3 Modelling parameters 


Variable Value 
Frequency of beam f = wo/(27) 14 kHz 
Quality factor Qo 43.2 


749.3702 m/(Ks)* 
1.0066 - 103 1/s 
4.2588 - 107 KQ2/sv2 


Heater resistance R 25 Q 


Transfer factor œ (temperature to deflection) 


Time constant £ (temperature change) 


Transfer factor (voltage to temperature) y 


Time constant of high-pass filter t 107° s 


Calibration factor k (piezoresistive sensing and | 10° V/m 
pre-amplification) 


Bias voltage upc —200 mV 
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Fig. 11 Fixed point x* in dependence on bias voltage upc as given in Eq. (7) (solid curve) and by 
direct simulation of Eqs. (4) (dots) for a = 0.6. Parameters as in Table 3 


B for temperature changes, the transfer efficiency y from actuation voltage into 
temperature changes, and the resistance R of the actuator. They are summarized in 
Table 3. 

Setting all time derivatives in Eqs.(4) to zero, we find the fixed point of the 


cantilever sensor: 4 
ay (UDC 
a(S). (7) 
apoB \ R 


A fixed point is a property of dynamical systems, which corresponds to the stationary 
state of the system. The dependence of the fixed point x* (i.e. the steady state or DC 
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Fig. 12 Left (fixed point): Simulated time series of the deflection for a = 0.6. Right (oscillations): 
Simulated time series of the deflection for a = 0.9. Initial conditions x(0) = 1 nm. Other parameters 
as in Table 3 


value of the deflection) on the bias voltage upc is depicted in Fig. 11. Depending on 
the feedback strength a, the dynamical behaviour of the system changes. Exemplary 
timeseries are shown in Fig. 12 for feedback strengths a = 0.7 (top) and a = 0.9 
(bottom), i.e., below and above the bifurcation point, respectively. In the top panel, 
the timeseries quickly approaches the value of the fixed point starting from the initial 
condition x(0) = 1 nm (cf. Fig. 11). Above the bifurcation, the fixed point is unstable, 
and thus, the dynamics do not reach the steady state. Instead, an oscillation emerges 
with an amplitude of 0.35 um. 

For further insight into the full dynamics for the case of oscillations (after bifur- 
cation), Fig. 13 depicts all four variables of Eqs. (4) and (6): x, y = x, 0, and uac. 

The transition from the fixed-point behavior to oscillatory dynamics occurs via a 
Hopf bifurcation. Beyond the critical point, a square-root dependence of the oscil- 
lation amplitude on the bifurcation parameter (here: a) provides a good indication 
of this bifurcation. Indeed, this is shown in Fig. 14, which depicts the amplitude of 
oscillations in dependence in the feedback strength a. The amplitude is calculated 
as max[x(t)] — min[x(f)] with ¢ € [0.95 s, 1 s] starting from an initial condition 
x(0) = 1 nm. The bifurcation occurs for a critical feedback strength agit % 0.8. 


4.3 Adaptation 


Since the sensor dynamics can be easily tuned by the real-time feedback, adaptation 
can be implemented by changing dynamically the feedback parameters: feedback 
strength a and bias voltage u pc. To implement an automatic adaptation of the sensor, 
the amplitude of the sensing signal is used for control of the adaptation. Conceptu- 
ally, one of the feedback parameters (here: feedback strength a) switches to another 
value, if the amplitude is larger than a pre-defined threshold. Thereby, two adaptation 
variants were implemented, which differ in the reset of the feedback parameter to its 
initial value: 
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Fig. 13 Numerical results (Oscillations): Time series of the deflection for a = 0.9. The different 
panels show all four variables of Eqs. (4) and (6). Initial conditions x (0) = 1 nm. Other parameters 
as in Table 3 
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Fig. 14 Numerical results: Amplitude of oscillations in dependence on a for an initial condition 
x(0) = 1 nm. Other parameters as in Table 3 
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Fig. 15 Schematic of the sensing system including the cantilever, adaptive feedback (blue dashed 
box), and thermal actuator. The adaptation includes an envelope generator, a switching-trigger, and 
the switching of the feedback strength a or bias voltage upc 


e Variant 1: The reset is done after a pre-defined time interval, independent of the 
signal amplitude. In this case, the response of the system resembles an event-based 
or spike-like response of the system to sound input. Thereby, with a fixed threshold, 
a spike-rate based encoding of the sound amplitude is possible. 

e Variant 2: The feedback parameter is reset to its initial value, if the amplitude of the 
sensing signal becomes smaller than a second threshold. This variant enables an 
automatic gain control and dynamic range enlargement. For small sound pressure 
levels, the nonlinear response of the system has a larger gain than the linear one, 
whereas for larger sound pressure levels the gain for the linear response will be 
higher. Thus, operating the system in general in the nonlinear mode and switching 
to linear mode for larger sound pressure levels can ensure a sufficiently high 
gain for the complete input range. Note that the nonlinear operation increases the 
dynamic range for small SPL by approximately 6dB compared to the linear mode. 
Furthermore, the response of the system resembles the short-term adaptation at 
the hair cell-auditory nerve synapse for constant sound input (cf. Sect. 2). 


Both variants enable a highlighting of the sound onset, which might improve the 
performance in processing tasks like localisation. These are typically based on the 
time difference of sound detection between two microphones. Thus, highlighting the 
sound onset might increase the efficiency of the processing for localisation. 

A schematic of the implementation for both variants is shown in Fig. 15. It consists 
of (i) an envelope generator, which determines the envelope from the sensing signal, 
(ii) a switching-trigger stage that is responsible for comparison of the envelope with 
the pre-defined threshold(s), and (iii) a switching stage to set the feedback parameter 
to the respective value. Thereby, variant 1 was implemented in the FPGA, which 
is used for feedback calculation. Variant 2 was implemented using analog circuits, 
whereby the circuits incorporated additionally the feedback calculation [97]. For 
this implementation, the STEMLAb board, including its ADC/DAC and FPGA as 
described in Sect. 4, was not used. 

Figure 16 shows the measured time series of the sensing signal in case of variant 
1 (left graph) and the envelope of the sensing signal for variant 2 (right graph). For 
variant 1, the feedback parameter a was varied between dy = 0.75 (nonlinear mode) 
and a; = 0 (linear (passive) mode) if the envelope of the sensing signal reached a 
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value of 200mV. In the linear mode, the sound input is not large enough to yield a 
sensing signal above noise level. Thus, a spike-like output is observed for this variant 
of adaptation. 

The time interval between two spikes is composed of two components: first, the 
response time Tı of the sensor until switching, and second, the off-time t2, i.e., the 
pre-defined time interval, for which the feedback parameter is kept at its lower value. 
Since the response time or transient behaviour of the sensor depends strongly on 
the input amplitude and the value of the feedback parameter a, a sound amplitude- 
dependent spike rate is observed, yielding an encoding of the sound amplitude by 
the spike rate, and the spike shape can be tuned by changing a. Such a spike-based 
output can improve the efficiency of the sensing system, because it reduces energy 
consumption, since feedback is turned off between spikes, and it reduces the amount 
of data, which needs to be sent to the processing unit, if only the spike times are 
transferred. 

In the second variant [97], the feedback strength a is switched from a high value 
(a = 0.6, nonlinear mode) to a lower value (a = 0.2, linear mode), if the threshold 
is reached. The reset to the initial values occurs, if the envelope of the sensing signal 
decreases below a second threshold. If a constant sound input is applied, the sensor 
will react first with high sensitivity (indicated by the peak in the time series), which 
decreases after switching to a lower level (represented by the plateau in Fig. 16 right 
graph). Comparing the change of the amplitude at the peak and at the plateau for 
different sound pressures (i.e. different driving voltages for the loudspeaker), the 
change from nonlinear characteristics (at peak region) to linear characteristics (at 
plateau region) is clearly visible. This variant has three main purposes: first, it can 


ieee a a 
f hia fh) siaaa a Libel Un 
Mri pardan e Ad 
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sensing signal 
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time [s] 


Fig. 16 (Left) Measured sensing signals for constant sound input (100 mV driving voltage loud- 
speaker) obtained from experiments with FPGA-based implementation of adaptation variant 1, 
switching feedback strength a between a; = 0.75 and ag = 0 (upc = —200mV). Here, the feed- 
back strength is kept at its lower value for a constant time interval t2 before resetting it to the 
high-sensitivity regime. The spike rate of this spike-like response of the sensor system depends on 
the sound-amplitude dependent part tı and the fixed time interval t2 for reset. (Right) Envelope of 
sensing signal for adaptation variant 2 and constant sound input between 0.14s and 0.7 s. Thereby, 
a was switched from 0.6 to 0.2 while upc was kept constant at —200 mV. Different sound pressures 
were applied by varying the amplitude snd for the loudspeaker (see legend). From [97] 
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increase the dynamic range, for which the gain (or sensitivity) is above a certain 
threshold. This is particularly important for large sound pressure amplitudes, since 
in this regime the nonlinear characteristics yield a small gain. Second, the overall 
shape highlights the onset of the sound, which might be important for processing 
tasks like localisation or recognition of speech. Third, the power consumption of the 
system might be decreased, since the feedback amplitude is reduced for the linear 
mode. Additionally, the shape of the response resembles the measured response at 
the synapse between hair cell and auditory nerve. However, it is until now, not clear 
if this adaptation is caused by biological restrictions (e.g. depletion of vesicles) or if 
it has a functional purpose for the subsequent processing. 


5 Conclusions 


The goal of this chapter has been two-fold: (1) to provide an overview of acoustic 
sensing from a biological point of view and (2) to present experimental and numerical 
results on an adaptable acoustic micro-electromechanical systems (MEMS)-based 
sensor. The mathematical model reflects the key experimental features. Besides the 
accessible sensor signal, the model also includes a variable for the thermal actua- 
tion of the electro-mechanical cantilever. Direct simulations indicate that the device 
undergoes a Hopf bifurcation. Dyamically, this corresponds to the transition from a 
fixed point to self-sustained oscillations. The response to different sound pressure 
levels was measured experimentally, where we identified passive, active linear, and 
active nonlinear modes. In the final parts of this chapter, we highlighted potential 
applications of the considered MEMS. We elaborated on experimental implemen- 
tation of spiking functionality via feedback strength switching that was realized by 
an field programmable gate array (FPGA). We also demonstrated the possibility of 
sensory adaptation, which is based on dynamic switching of the feedback strengths. 
This will pave the way, for instance, for frequency-selective amplification of sound 
signals. 
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Abstract Decision-making belongs to one of the most prominent principles in the 
nerve system of living species. The crucial trade-off between time to decide (speed) 
and accuracy is essential for a creature to react to environmental stimuli and most 
importantly, relevant for survival. In-depth studies of coherent motion tasks in pri- 
mates, based on two-choice decision experiments, led to a profound understanding 
of neuronal information processing and even bridged the gap to behavioural psy- 
chology. Three fundamental processing stages are needed to successfully perform a 
two-choice decision experiment based on a saccadic eye movement task: the neu- 
ronal representation of the sensory signal, the integration of the incoming stimuli 
and the comparison of the accumulated information to a threshold for a final deci- 
sion. In accordance to the results of decision-making experiments with primates, 
an analogue electronic decision-making circuit was developed. Our concept study 
includes a LED-matrix showing moving light dots (i.e. the tasks screen in primate 
experiments), an array of photo diodes (i.e. retina of the primate, sensory evidence), 
a Hassenstein-Reichardt Detector (HRD) based motion detection (i.e. a correlation- 
type motion detector with a high biological relevance, based on an inhibitory circuit 
and delay lines) and finally a signal integration circuit. The latter stage represents 
the decision-making unit and is realised through an inhibitory mutual coupling of 
the accumulated signals. The entirely analogue circuit is based on integrate and fire 
relaxation oscillators to mimic firing neurons. We would like to emphasise that the 
presented real time decision-making system is closely related to biological infor- 
mation pathways in nerve systems. Neither a digital computer nor programming is 
necessary for decision-making, which might be an interesting approach for real time 
data processing in autonomous robots. 
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1 Introduction 


The remarkable interaction performance between biological nervous systems and 
their environment is a result of million years of evolution explained by Darwin- 
ism [1]. Indeed, humans, mammals and even simple life-forms as invertebrates are 
well adapted to permanently changing surroundings and only appropriate reactions 
ensure survival. Exploring the whole signal pathway from sensory transduction, 
decision-making neuron ensembles and the final response via motor neurons and 
muscles is the challenging task in neuroscience. Although a tremendous progress 
in the understanding of biological nervous systems can be noticed, there still is a 
considerable lack to understand all principles of information processing from the 
molecular length scale to the system size. In other words the “brain code” is not 
yet understood. Consequently, up to today it is impossible to engineer bio-inspired 
neuromorphic systems, which mimic the signal pathways of living species. The real 
benefits of biological information processing when it comes to cognitive tasks or 
pattern recognition together with extreme power efficiency, are still unattainable in 
their technical pendants. Instead of trying to solve all problems at once, it is much 
more efficient and pragmatic to mimic specific features of biological nerve systems 
by focusing on simpler and more visible goals. This strategy has been applied very 
successfully in the past [2]. We follow the same path. The presented work focuses 
on central aspects of biological two-alternatives decision-making and its neuromor- 
phic realisation in a compact battery-powered analogue circuitry, which might be of 
relevance in the field of robotics. 

The paper is organized in the following way: starting with an example of decision- 
making in daily life, models based on psychophysical experiments are shortly sum- 
marized. Hereafter, the most relevant model for this work, the leaky competing 
accumulator, is explained by means of a circuit block diagram. This scheme is trans- 
ferred to an analogue circuit comprising a Hassenstein-Reichardt motion detector, 
relaxation type oscillators and delay-lines. Subsequently, experimental results based 
on moving dot tasks and the performance of the technical decision making system are 
presented. Finally, similarities and deviations between the bio-inspired electronics 
and the biological motivated, leaky competing accumulator are discussed. 

Studies of decision-making encompass a broad range of research fields such as 
neuroscience, psychology, statistics, economics and robotics [3]. Essential ingredi- 
ents of a typical decision making task are illustrated in the comprehensible cartoon 
in Fig. 1. 

In a board game of dice you (represented by the yellow pawn in Fig. 1) may face 
two possible paths by reaching an intersection and a decision has to be made [4]. 
The right path is shorter (less steps) compared to the left one and you might be able 
to reach the goal faster but the path includes risky fields (labeled in red). With bad 
luck your pawn has to turn back to the start, with the consequence that you might 
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Fig. 1 Board game cartoon 
to illustrate a binary decision 
task. The yellow pawn could 
take the left path at the fork, 
i.e. many steps but a save 
track towards the goal or the 
right path, which means less 
steps but a higher risk 
because a hold on the red 
areas will lead to a reset. 
[adapted from [4]] 


lose the game. Decision-making in our daily live, even under much more serious 
consequences, is omnipresent. How can a final choice, a decision, be explained in 
terms of neuronal signal pathways in our brains or in mammals in general? This 
question is at the heart of decision-making research and is discussed over the last 
decades [3, 5-10]. In addition we would like to emphasize that decision-making is 
often made under limited time and leads to a tradeoff between decision quality and 
decision-making time, also called accuracy—speed task [11-13]. 

Two major categories of decision-making models have been developed. The first 
category applies to complex decisions among valued alternatives [7, 14, 15]. Here 
the time for a final decision could take minutes or even much longer. The second 
category, the so-called sequential-sampling type of models, applies to binary or 
multiple perceptual and memory related tasks [3, 8, 16]. In contrast to the valued- 
based decisions, the response time is typically in the order of a few seconds. In 
Fig. 2 the taxonomy of the main decision-making models belonging to the second 
category (perceptual tasks) are summarized. The models are explained by the simplest 
decision-making process, i.e. to choose one of two-alternative evidences, let’s say left, 
L, or right, R. Although neurobiological studies showed that these two model classes 
for decision-making (valid (not subject in Fig. 2) and perceptual) are linked to some 
extend [6, 9, 15], we will focus exclusively on two-alternatives perceptual decision- 
making models. Sequential-sampling models are divided in further subgroups, such 
as random-walk models and diffusion processes on the one hand and accumulator 
and counter models on the other hand [8]. For random-walk models as well as for 
accumulator/counter models it is assumed, that noisy input stimuli are accumulated 
(integrated) over time until a threshold criterion is reached, representing the final 
decision. Whereas, in random-walk models evidence is based on accumulating a 
single total, accumulator/counter models evidences are integrated over two processes 
separately, i.e. in case of two alternatives a kind of a distinct race towards both of 
the two thresholds. 
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Fig. 2 Taxonomy of perceptual decision making models. a Random step models. Here the differ- 
ence (e.g. left L signal evidence—right R signal evidence) between two alternatives is plotted versus 
time. An upper and lower threshold defines the final decision. Subgroups are the random-walk model 
and the diffusion type Wiener and Ornstein-Uhlenbeck models; b in accumulator-and counter-type 
models, the two evidences are separately integrated over time. Nonetheless an inhibitory, cross- 
wised coupling of the two sensory evidences assures a distinct final decision. Subgroups are the 
accumulator, the leaky competing accumulator and the Poisson type model [see [8] for more details]. 
In this work an analog circuit is realized in accordance to the leaky competing accumulator model 


After this overview about theoretical models of decision-making, a short survey 
of neurobiological experiments, in the context of perceptual decision-making with a 
focus on dot motion discrimination tasks, is presented [8]. 

Since the first experimental attempt by Newsome et al., visual-saccadic decision- 
making became one of the most important methods to study visual signal pathways 
in primates [5, 17, 18]. For example, in so-called coherent motion (two-alternatives) 
sensory-motor tasks, the experimental procedure is as follows: A dynamic ensemble 
of dots is displayed on a screen. Some of these dots move randomly, whereas others 
move either coherently to the left, L, or to the right, R. A trained primate (mostly alert 
rhesus monkeys) observes the scenario on the screen and makes, after a response time 
(typically 2 s), a decision, i.e. a saccadic eye movement to the left or right, indicating 
its choice (decision) in which direction most of the coherent dots previously moved. 
Simultaneously to the monkey’s decision-making, activities of visual cortical neurons 
are detected in vivo. A clear correlation between the monkey’s behavior and neuronal 
activity was found. Later on, other psychophysical experiments e.g., the oddball task 
and the dot separation task were developed [8] and extended to non-invasive methods, 
such as MEG (magneto encephalography), fMRI (functional magnetic resonance 
imaging) and EEG (electro encephalography) [19]. 

As raised above, decision-making is a biological process, which translates noisy 
sensory information via perception and cognition into an action. So finally the motor 
system executes the decision made by higher brain functions and represents the 
weighting of the nerve system and takes the possible alternatives, set by the envi- 
ronment and its own goals, into account. We would like to emphasize, that reward, 
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Fig. 3 Areas of the me 
oculomotor system involved 
in decision-making are 


exemplarily shown in a FEF —es LIP 


sketched brain of a primate. 


For decision making 
neuronal correlates such as, 
the middle temporal area MT 


(MT also called V5), lateral 

interparietal area (LIP), the 

frontal eye field (FEF) and SC 
the superior colliculus (SC) 

are involved 


in form of stored information in the nerve system, i.e. the former experience of the 
creature, has also considerable influence in the decision process but is not considered 
in this work [15, 20]. 

Which neuronal correlates are responsible for decision-making? For two-choice 
discriminator (saccadic eye movement) tasks, it becomes clear, that oculomotor areas 
including the middle temporal area (MT also called V5), lateral interparietial area 
(LIP), the frontal eye field (FEF) and the superior colliculus (SC), are involved. 
More details about the interacting neuronal areas can be found in Refs. [8, 21-23]. 
In Fig.3 these areas and their approximate locations in a non-human primate brain 
are depicted schematically. 

It is believed that neurons in the area LIP mediate a simple decision process by 
integrating the sensory evidences over time towards a threshold, i.e. a criterion to 
commit one of the two possibilities. In other words, the decision process is a race 
between two competing alternatives. Binary, opposite movements in decision making 
are simultaneously elusive because a single motor response is required. Hence, mod- 
els based on not fully independent sensory systems, including a mutual inhibitory 
component between the two signal pathways, were developed. The principles reflect- 
ing essential stages of the saccadic eye movement experiment are shown in Fig. 4a 
[24, 25]. A primate is facing a display to perform the coherent motion task. The 
behavioral task is illustrated in the same figure. Neuronal direction-sensitive motion 
detectors in the primate brain extract the visual strength of the moving dot ensembles. 
In more detail, the signals for dots, moving to the left and to the right are accumulated 
(integrated over time) by combined neurons. Both signal pathways are coupled via 
inhibitory neurons. Finally the output neurons indicate the decision and trigger motor 
neurons for the final saccadic eye movement (target T; —left or target T,— right). 

In accordance to the behavioral task as shown in Fig.4a, a neuromorphic cir- 
cuitry was build and the corresponding circuit design is shown in Fig. 4b and will be 
explained hereinafter. 
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Fig. 4 a Sketch of the biological perceptual decision-making experiment. The primate facing a 
dot display performs a two-alternatives discrimination task defined by an ensemble of moving dots. 
The final saccadic eye movement reflects the decision whether the majority of the dots has been 
moving to the left or to the right; b block diagram of the biologically inspired perceptual decision 
making process of a two-alternatives discriminator task. From left to right and in correspondence 
to the biological pendant the following units are shown: LED (8 x 8) matrix to define computer- 
controlled moving dot tasks, a photo diode receptor unit as a retina pendant (a 5 x 5 matrix), a 
correlation-type Hassenstein-Reichardt motion detector (HRD), two integrating units for the two 
pathways, cross-wised linked inhibitory neurons and two accumulation units. The corresponding 
biological units, the MT areal and the LIP-areal are marked [7, 26] 


2 Materials and Methods 


2.1 A Bio-inspired Block Diagram 


The essential parts (left to right in Fig.4b) are a computer controlled LED-matrix 
(representing the task screen in biological experiments), an array of photo diodes 
(i.e. retina of the primate, sensory evidence), motion detection (i.e. a correlation-type 
motion detector with a high biological relevance) based on an inhibitory circuit and 
delay lines) and finally a signal integration circuit for decision-making. The LED- 
display is connected to a digital computer via a microcontroller interface, in this 
way different computer-controlled tasks can be realized on the display. The motion 
detector is based on the Hassenstein-Reichardt motion detector (HRD) model, which 
was developed in the 1950‘s to explain signal pathways in insects, such as the fly 
[27-32]. The concept was further developed by Barlow and Levick [33]. Before 
continuing with the entire decision making system, the HRD will be described in 
more detail in sequel. 
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2.2 The Hassenstein-Reichardt Motion Detector (HRD) 


The HRD is a correlation-type system and comprises in a mitror-like fashion two 
signal pathways including the photoreceptors P; 2, two delay lines t1,2 two (non- 
linear elements) NL; 2 (e.g. multipliers or a vetogate) [27, 33-35]. Finally the signals 
of the two pathways are subtracted. To illustrate the function of the HRD, an object, 
O, moving with a positive velocity (i.e. from left to right) is shown in Fig. 5a. At 
first the photoreceptor Pı detects a visual signal if the object passes by. The signal is 
split into two parts, the first serves as the input for the delay line tı, while the second 
part of the signal represents the input of the non-linear element NL2. The qualitative 
steady-state response of a HRD is shown schematically in Fig. 5b. For a static object 
(v = 0 cm/degree) the output is zero. For objects moving either from left to right or 
vice versa, the HRD delivers a maximum/minimum response signal, respectively. 
This allows an insect (such as Chlorophanus or Drosophila) to distinguish between 
objects with different directions of motion. For larger absolute velocities beyond the 
maximum/minimum of the object, the signal response gradually vanishes for both 
directions. 


Fig. 5 a Principle 
components of a HRD, 
comprising two 


a 


photoreceptors, two delay Y Y 
lines (t1,2 two non-linear t = p- | 
elements NL} 2). The a So t 
information pathways are T : JO . T 


cross-linked; b schematic 
response curve of a HRD 


È ES y a 
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Fig. 6 View graph of the HRD circuit with an input stage comprising five photoreceptors, five 
input oscillators (layer I) and five delay lines (layer II). Six output oscillators (layer II) are used 
to indicate the decision between the two-alternatives (L or R). The red and green colored arrows 
correspond to object movements from left to right or right to left, respectively. On the right hand 
site a graphical symbol of the HRD is shown 


A more detailed layout of the developed HRD is shown in Fig. 6. The visual input 
received from the five photo receptors (photo diodes) is connected to five oscillator 
units Os (layer I). The first layer is connected via five delay lines (marked by t) to the 
outputs which are further connected to another layer II comprising six oscillators. 
One part of layer II (the three oscillators marked in red) and the other part of layer 
II (marked in green) are indicators for motions from left to right and vice versa, 
respectively. It is worth to mention that one path in the middle is split into two 
subsequent oscillators on each site (one red and one green). The relaxation type 
oscillators offer the opportunity for excitatory and inhibitory inputs to realize pivotal 
neuronal functionalities and will be explained in more detail below [4, 36]. 


2.3. The Relaxation-Type Oscillator 


A spiking neuron can be represented by an oscillator model [4, 37]. How have the 
oscillators and the delay-lines been realized electronically? The circuit diagram of 
the implemented relaxation-type oscillator including the excitatory (MOSFET Mo) 
and inhibitory (MOSFET M;) coupling scheme and that of a delay-line are shown 
schematically in Fig. 7a and b, respectively. 

The relaxation-type oscillators consist of a programmable unijunction transistor 
(PUT). (ON Semiconductor, Ebers, 1952, Floyd 1999). PUTs belong to the class of 
silicon rectifier devices and have four (npnp) alternating regions (see Fig. 8a). By 
applying a bias voltage between the anode and cathode, the PUT exhibits an J — V 
curve with a negative differential resistance region as shown in Fig. 8b. Programmable 
means that the gate Voltage, VG, applied to the gate terminal, modifies the oscillator 
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Fig. 7 a Circuit scheme of a Vv 
programmable unijunction 
transistor (PUT) based 
oscillator. Two MOSFETs, = In 
Me and Mi, are connected to (Ir, Yr, i e 
the gate resistance RG and Lt 
mimic either the biological Knot , Knot, 

p Resch 2| 


motivated excitatory (via — + 4 T 
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coupling scheme of neurons. 
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pulse at the cathode C of the 
PUT can be used to trigger 
other oscillators or 
delay-lines (signal out). The 
photo diode parallel to RG V 
represents one pixel of the 

receptor matrix (input 

oscillators in layer I in 
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frequency. The relaxation-type oscillator (see Fig. 7a), works as follows: The capac- 
itor C is charged through the resistor R leading to the voltage Vc at knot 1. By 
reaching the voltage V;, = Vg + 0.7 V defined by the voltage divider Rg, Rı and 
the bias voltage VB, the PUT “fires”, i.e. the charge of the capacitor C is discharged 
via the cathode terminal to ground and hence the cycle perpetually repeats. Typical 
voltage traces of Vc of the capacitor C (at knot 1) and the gate terminal Vg (at knot 2) 
are depicted in Fig. 9. For low frequencies (<20 Hz) a LED between the cathode and 
ground (and parallel to the cathode resistor Rc) can be used to visualize the oscil- 
lations. An extended circuit was used to modify dynamically the gate resistance Rg 
and mimic excitatory and inhibitory properties of neurons. For those functions, two 
MOSFETs were integrated, one (Me) which induces the oscillations (excitatory) and 
another (M;) which suppresses the oscillations. The photo diode was implemented 
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Fig.8 a Schematic of the np regions and terminals of the PUT 2N 6027 [39]. Essential is the internal 
connection of the n; and p? regions to realize non-equilibrium charge carrier effects, necessary for a 
negative differential resistance region (NDR) in the J — V curve [40]. b J — V curves measured for 


positive bias voltages between the anode and cathode. The negative differential resistance regime 
is clearly visible 
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Fig. 9 Upper view graph: Characteristic relaxation-type voltage-time traces of the Vc (corresponds 
to knot 1 in Fig. 7) which represents the charging of the capacitor via the resistor R. Lower view 
graph: When the threshold voltage V;, (defined by the voltage divider R1, Rg and the bias voltage) 
is reached, the PUT fires (relaxes), which results in a short voltage drop at the gate terminal Vg 


(corresponds to knot 2 in Fig. 7). For this particular set of parameters the oscillator frequency was 
approximately 25 Hz 
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for the oscillators shown in layer I (Fig. 6). This input oscillators in layer I act as an 
opto-electrical interface and emulate the function of a the retina. The oscillators of 
layer I start to oscillate only, if the illuminated photo diodes receive a light stimuli 
whose intensity is high enough. We would like to emphasize, that for the basic ver- 
sion of the PUT (i.e. without the photo diode and the MOSFETs Me and Mi at the 
gate terminal, Fig. 7a), the parameters were chosen in a way to suppress oscillations. 
This is the case when the voltage Vc does not reach the threshold voltage Vth of the 
PUT. Sole in the particular case where the external stimuli is large enough oscilla- 
tions are observable. This stimuli is either optical for layer I or electrical for all other 
oscillators at the excitatory input (M_) in the absence of any inhibitory signal (M;). 
Then the relaxation pulses of the oscillators are easily visible via the LED between 
the PUT’s cathode and ground. 


2.4 LED-Matrix Chaser as a Dot-Task Display 


In Fig. 10 a picture of the entire decision making circuit is shown. The LED-matrix 
was face-to face with the 5 x 5 five photo receptor array. Since the matrix corresponds 
not one-to-one (8 x 8 LED to5 x 5 photo receptors), only the inner area of it was used 
as an active part. For the microcontroller interface, an Arduino was used. To program 
the device it was connected to an laptop using a USB. For programming, tools of the 


Fig. 10 Photograph of the entire decision making system. The LED display was face-to face 
with the 5 x 5 five photo receptor array. The circuit consists of five main subgroups: (1) LED- 
Matrix controlled via Arduino, (2) Layer I Oscillators, (3) HRD, (4) Layer II - Outcome HRD, (5) 
Accumulation Stage 
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Arduino software were used. For a typical dot-task presented to our decision-making 
system, the LED matrix was programmed as light chasers for each column. This 
means for example that for three adjacent LED columns the LEDs simultaneously 
chased from left to right and for the other two columns vice versa. The frequency, 1.e. 
the repetition time for one cycle in a column was ca. | s. Under this imbalance condi- 
tion of moving dots (more left to right then right to left) and similar to physiological 
motivated dot tasks, we tested the decision-making capability of the system. 


3 Results and Discussion 


3.1 Entire Electronic Decision Making System 


The aforementioned components of a technical decision making system are summa- 
rized for one column of photo-detectors and sketched in Fig. 11. The two outputs of 
each HRD (representing L and R) are connected to the left and right accumulation 
stage. Each accumulation stage consists of one capacitor integrating the evidence for 
the right and left direction of motion, respectively. The diodes between the HRDs and 
the accumulation stages suppress coupling between the HRD oscillators and the dis- 
charging of the two capacitors. The voltages (Vz and Vp) of the two capacitors Cz and 


moa tooo LLILITT Shy et oe 


| HRD | | HRD | HRD | HRD | HRD | 


Fig. 11 Entire layout of the biologically-inspired perceptual decision making circuit. At the stages 
S; to Sq signal traces were acquired 


Kej 
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Cr are used to compare the two evidences. To indicate the final decision the voltages 
are encoded by two indication oscillators. To get the inhibitory coupling scheme as 
shown in Fig. 4, the encoded Vz and Vp serve as inhibitory inputs for the right and left 
indication oscillators, respectively. This assures the essential mutual influence of the 
two evidences as described by the leaky competing accumulator model (see Fig. 2). 


3.2 Result of Decision-Making 


To illustrate the function of our decision-making system, including the voltage traces 
at different stages, we defined a clear and simple task via the LED matrix. Only one 
column was used for the light chase and programmed in a way, that the direction of 
the light stimulus alternates between left to right and right to left establishing an sym- 
metric input. The complete period was approximately 1 s. For this task the following 
voltage traces were measured at the stages S4 to S4 as labeled in Fig. 11. The results 
are shown in Fig. 12. In Fig. 12a the encoded light stimulus at the oscillation layer I 


J 


Fig. 12 a The five voltage 
versus time traces of layer I 
oscillators connected to one 
column of the photo diode 
array (see Fig. 6). The 
pattern represents the LED 
light chase in one row; b the 
six voltages traces of the 
Hassenstein-Reichardt 
detector (HRD) (see Fig. 6, 
layer II). c Layout of the 
photo-diode array of Layer I. 
The voltage traces of De ee 
oscillators connected to the F[— Osd3 — Osrl2— Osr 1 — Osir1 — Osir2— Osir3 
marked diodes are depicted 
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Fig. 13 Typical results of the decision making circuitry for a dot discrimination task. In the upper 
diagram the voltage versus time traces of stage S3, which correspond to the charge state of the 
two capacitors (see Fig. 11), are shown. Each trace represents an indicator for the strength, i.e. the 
number of dots moving to the left (movement R to L, black line) or right (movement L to R, red 
line) direction. The lower diagram depicts the encoded information in form of firing rates of the 
two output oscillators, used as a negative feedback 


(stage S1) , i.e. the input for the HRDs is shown. The five spike sequences represent 
the sensory transformation of the optical pattern. In more detail, the sequential spike 
arrangement reflects the chronology of the programmed chasing light cascade which 
served as inputs for the correlation-type motion detectors. The output after the HRDs 
is depicted in Fig. 12b (at stages S2) exhibits the two motion directions. 

In Fig. 13 the voltage traces at S3 and S4 are depicted. S3 represents the charge 
state of each capacitor. The time evolutions in the upper diagram reflect the accu- 
mulation of evidence during a decision process. At the beginning of the task (t = 
0 s) the two signals increase simultaneously, because the system recognized evi- 
dence for the two alternatives. In the inset the time interval up to 2 s is enlarged. 
The race of the two signals is clearly visible. For times larger than 0.7 s, evidence 1 
(black) increases, whereas evidence 2 (red) decreases. The later is due to the negative 
feedback, established by the discharge MOSFET at the accumulation stage S3 and 
reflects the implemented inhibitory function. We like to emphasize the similarity of 
these results to those, predicted in the biological motivated accumulator and counter 
models shown in Fig. 2b. The charge state is encoded in the firing rate detected at the 
cathodes of the two oscillators at stage S4. This ensures a counteracting response, 1.e. 
one evidence increases at the expenses of the other evidence during decision-making. 
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4 Conclusion 


In summary, we presented a bio-inspired analogue circuit for a two-alternatives per- 
ceptual moving dot task, known from physiological based discrimination experi- 
ments. Pivotal neuronal correlates of oculomotor areas in primates, such as the mid- 
dle temporal area (MT also called V5), lateral interparietal area (LIP), the frontal eye 
field (FEF) and the superior colliculus (SC) known to be involved in decision-making 
discriminators task, were mimicked by an analogue circuit. Based on the principle 
of a Hassenstein-Reichardt motion detector, relaxation-type oscillators, delay-lines, 
a LED-matrix and a photo-diode matrix, simple two-alternatives decision tasks were 
solved by our system. The circuit processes in real time and no computer is involved 
(we exclude here the laptop/microcontroller used to define the motion task). Rather 
than developing the system to break the world record in decision-making performed 
by a technical system, we focused on a closely bio-inspired version. Fundamental 
principles such as spiking neurons, inhibitory and excitatory coupling schemes and 
the Hassenstein-Reichardt motion detector were emulated. 

In the future it might be possible to include a reward in form of memristive devices 
which change their state. Moreover studies as the system response-time versus the 
task difficulty can be performed [41, 42]. We hope that our approach makes a relevant 
contribution to the field of robotics based on neurobiological principles. 
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Abstract The rapidly expanding field of biomimetics emulates biological princi- 
ples, distilled through evolutionary time, to solve problems in technology, medicine 
and material science. Information processing in neuronal circuitry of biological mod- 
els may offer guidelines for future highly efficient computing systems exhibiting, as 
for example, low power consumption accompanied by excellent pattern recogni- 
tion capabilities. Here, we consider the visual information processing in the nervous 
system of the box jellyfish Tripedalia cystophora as template for pattern recogni- 
tion hardware to operate self-guiding robotic vehicles or automated driving assis- 
tants. LTspice XVII simulations of coupled relaxation-type oscillators, based on 
programmable unijunction transistors (PUTs), enabled the emulation of basal visual 
functionalities of the T. cystophora central nervous system. The transfer of such 
simulations in real hardware circuits including possible limitations are discussed. 
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1 Introduction 


Biological model systems are studied to understand distinctive biological processes 
and mechanisms. Along these lines various species are investigated because it is 
believed that explorations made in the model organism will render insight into the 
working principles of other organisms, including humans. Well known examples 
which have been extensively studied are mice, rats, zebrafish, the fruit fly Drosophila 
melanogaster and the nematode worm Caenorhabditis elegans [1—4]. Despite the 
immense differences in the details, it is believed that the most basal mechanisms 
of development are recapitulated in all animals and are evolved from the earliest 
animal ancestors. Hence, the elucidation of a developmental process in one animal 
is often of importance for understanding development in another [5]. Despite the 
significance of model systems for comparative studies in biology, they may also 
present novel guidelines for the field of bio-inspired engineering. The goal hereby 
is to extract evolutionary shaped basal biological mechanisms from the point of 
power efficiency and sensitivity and transfer those to technology. Visual information 
processing in the relatively simple T. cystophora nervous system offers a unique 
opportunity to reverse engineer pattern recognition electronic hardware, and thereby 
circumvent the need for processing computers to run pattern recognition software. 
Fast responding pattern recognition hardware could support self-guiding robotic 
vehicles or automated driving assistants. Here, we consider the visual system of the 
box jellyfish and realize their optical communication principles in electronic circuitry 
comprising a set of coupled relaxation-type oscillators [6, 7]. In the first stage we 
simulate the circuitry by LTSpice XVII [8]. In the present theoretical study, coupled 
relaxation-type oscillators with a fixed topology enabled essential box jellyfish motor 
effector activation used in coordinated swimming, hovering, and turning [9]. 


2 The Box Jellyfish: Anatomy, Dynamic and Behavior 


The box jellyfish Tripedalia cystophora is emerging as a new model system for 
basic visual information processing. Photographs of the visual system and nervous 
system of T. cystophora are shown in Fig. 1 [10]. The T. cystophora model system 
is attractive due to its vertebrate like camera type eyes [11, 12], multiple distinct 
visual behaviors [13-16], and an experimentally tractable central nervous system 
comprising approximately 1000-1500 processing neurons [17, 18]. 


2.1 Visual system 


The visual system of T. cystophora, and box jellyfish in general, comprises 24 eyes of 
four distinct different types. Eight of these are vertebrate like camera type lens eyes 
complete with ‘Matthiessen’s ratio’ lens with graded refractive index, retina with 
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5 ring nerve 
+ rd 


Fig. 1 The visual system and nervous system of Tripedalia cystophora. a An adult specimen of 
the mangrove dwelling box jellyfish Tripedalia cystophora has a bell diameter of about 10 mm. b, 
c The eye-brain complexes (rhopalia) each carries 6 eyes, of four morphological different types, 
and the processing neural circuitry. The four rhopalia are suspended from the sides of the bell by a 
flexible stalk and weighed down by a crystal on the distal end. The epidermal stalk nerve conveying 
the motor signal is located in the medial side of the stalk. A circumnavigating ring nerve enables 
inter-rhopalial communication (a). PE: pit eye, SE: slit eye, ULE: upper lens eye, LLE: lower lens 
eye. Figure adapted from [10] 


338 J. Bielecki et al. 


opsin-based photoreceptors, and pigment screen to accommodate directional vision 
[12, 19, 20]. The eyes are distributed on four sensory structures termed rhopalia (see 
Fig. 1b and c), each of which is carrying two lens eyes and four ‘lesser’ eyes [12, 
20]. The rhopalium also carries the processing neuronal circuitry used to process 
the visual information collected from the eyes. The rhopalia are suspended from 
the sides of the animal by a flexible stalk, which contains the efferent epidermal 
motor nerve, and are inter-connected through a ring nerve that circumnavigates the 
medusoid bell (Fig. la) [21]. It is thought that the ring nerve is involved in inter- 
rhopalial communication and, at least to some degree, coordinates the contraction of 
the bell [22]. A calcium sulphate hemihydrate crystal [23] at the distal end ensures a 
constant vertical orientation of the rhopalium and thereby constant field of view of the 
lens eyes [15, 24]. The lens eyes are thought to be special purpose eyes modulating 
specific visual behaviors. The upper lens eye is directed out of the water and into 
Snell’s window. Snell’s window is a physical phenomenon created by the difference in 
refractive indices between air and water that visually compresses the 180° hemisphere 
above water into an approximately 97° underwater cone. The visual field of just below 
100° perfectly matches the upper lens eye to Snell’s window [12]. By adjusting the 
direction and angle of the visual field to accommodate visual information received 
from Snell’s window, the upper lens eye can detect the contrast line between the 
mangrove canopy and the open sky as a bright line on a dark background [24]. T. 
cystophora can thereby use terrestrial cues for long distance navigation [15]. Tidal 
currents constantly pose the risk of sweeping T. cystophora into the middle of the 
mangrove creek lagoon and subsequently to the open sea. Being swept away from 
the primary habitat would prove fatal to the animal since T. cystophora forage in light 
shafts created by sunlight shining through the mangrove tree foliage and between the 
prop roots [16], where their photo tactic copepod prey is abundant. Therefore, if the 
animal finds itself under the open sky, it will quickly turn and swim back under the 
canopy [15]. The lower lens eye is directed 60° into the water and with a visual field 
of approximately 170° scans the underwater environment for potentially dangerous 
objects [12, 13]. Ithas been shown that the lower lens eye overrides the upper lens eye, 
indicating that underwater input is regarded of higher importance than above water 
input [25]. Intuitively a sensible prioritization since collision with the prop roots 
of the mangrove trees poses an immediate danger of inflicting fatal damage to the 
fragile box jellyfish bell. As mentioned, the rhopalia are interconnected through the 
ring nerve and the overall behavior of the animal results from coordination between 
the rhopalia, prioritizing the rhopalium that detect changes in the visual environment 
[26-28]. 


2.2 Obstacle Avoidance Behavior 


When T. cystophora encounters an underwater obstacle it will perform 4-5 fast bell 
contractions, turn 120—180° and swim away (Fig. 2) [13]. This distinct visual behav- 
ior is modulated by the lower lens eye and is based on true spatial vision. The animal 
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Fig. 2 Obstacle avoidance behavior. a Tripedalia cystophora navigating the mangrove habitat must 
avoid collision with underwater object to prevent damaging the fragile bell. b When the box jellyfish 
encounters an obstacle estimated close enough to pose an immediate danger, the animal will initiate 
obstacle avoidance behavior. c Obstacle avoidance behavior comprises 4-5 fast bell contractions, 
turning 120-180° and swimming away from the obstacle 


shows a graded avoidance response to varying contrasts of the obstacles, higher 
contrast yielding greater response—presumably an inherent gauge of distance under 
water [14]. Under the canopy, T. cystophora explores the mangrove habitat in search 
of foraging opportunities in the light shafts (Fig. 2a). When the animal encounters 
an underwater object, it will estimate the distance by the contrast of the object com- 
pared to the surrounding water. When the contrast surpasses the response threshold 
(Fig. 2b), the obstacle avoidance behavior will be initiated, which, as mentioned, con- 
sists of 4—5 fast bell contractions and directional control of the velarium (Fig. 2c). The 
velarium is a membrane-like constriction of the oral opening of the box jellyfish bell 
that can be asymmetrically constricted to form directionally controlled jet propul- 
sion of the water flow created by the bell contraction [26]. This enables the animal 
to turn 180° within 2-3 bell contractions. The bell contracting motor output signal 
is generated by oscillatory, or pacemaker, cells located on the rhopalium at the base 
of the stalk [18, 29]. The pacemaker cells have never been unequivocally identified 
but are thought to be what has been described as “giant neurons’ in the literature [17, 
18]. Regardless of location in the rhopalial nervous system (RNS), the pacemaker 
cells produce a swim pacemaker signal that contracts the box jellyfish bell in a 1:1 
manner. The intrinsic pacemaker cell frequency is 3—4 Hz, which is suppressed by 
neuropeptidergic mediation [30] to approximately 0.5—1 Hz. The latter corresponds 
to leisurely swimming and foraging in the mangrove habitat. The duration of one bell 
contraction from fully expanded bell to fully expanded bell is approximately 250 ms, 
corresponding to an absolute physiological relevant maximum of 4 Hz. Usually the 
animals display swim contractions of less than 3 Hz even during obstacle avoidance 
events. 


2.3 Pacemaker Activity Represents Behavior 


The swim pacemaker signals can be recorded from the efferent epidermal stalk nerve 
(Fig. 3) by a suction electrode (extracellular electrophysiology). The pacemaker sig- 
nals are 45 ms signals with a profile easily discernable from regular action potentials 
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Fig. 3 Pacemaker signals predict the behavior of Tripedalia cystophora. a To record the motor 
neuron output signals (pacemaker signals) the stalk is transected at about 2/3 of the distance to 
the bell. b This section reveals the two medially located epidermal stalk nerves (EN) where an 
extracellular electrophysiological suction electrode can be attached. c With the electrode in place, 
visual stimulation of the lower lens eye (LLE) can be performed and the responding pacemaker 
activity recorded. d Intrinsic pacemaker signal activity recorded from the EN (red arrowheads). 
The pacemaker signals have significant longer duration (45 ms) than normal action potentials. 
ULE: upper lens eye. Figure adapted from [30] 


Crystal 


(Fig. 3d). Since swim contractions are the only means of locomotion in T. cystophora 
the pacemaker signals characterize the behavior of the entire animal. This correlation 
is prominent when presenting light-ON and light-OFF stimuli to the lower lens eye. If 
a rhopalium is kept in the dark for a few minutes the pacemaker signal frequency will 
stabilize around 0.5-1 Hz. If subsequently the lower lens eye is presented with a light 
ON stimulus, the pacemaker signal frequency will suddenly decrease and sometimes 
completely cease. Conversely, a light-OFF stimulus to a light adapted lower lens eye 
will result in a sudden increase in pacemaker signal frequency. Whereas the light- 
ON response is long lasting (scale of minutes), the light-OFF response is transient 
and lasts about 10 s [25]. The pacemaker signal response is directly related to the 
foraging behavior of T. cystophora. When the box jellyfish enters a light shaft (light- 
ON), where phototactic copepod prey is abundant, the animal will cease swimming 
and start fishing by passively sinking with outstretched tentacles. If T. cystophora 
inadvertently swims out of the light shaft (light-OFF) it will perform a number of 
high frequency swim bell contractions, turn, and try to relocate the light shaft [16]. 


Pattern Recognition in the Box Jellyfish Rhopalial Nervous System Mimicked ... 341 


2.4 Lens Eye Morphology and Activity 


Light entering the lens eye is first refracted by the lens and focused onto the retina. 
The lenses of the upper and lower lens eyes are of remarkable high quality and 
capable of producing images on the retina with high spatial resolution. However, 
the retinas of the lens eyes are displaced closer to the lens than the focal plane of 
the lens. This severe under-focus is thought to remove fine image details to avoid 
overloading the processing circuitry. With this under-focus the receptive fields of the 
photoreceptors are 15—20° depending on their location in the retina [12]. The upper 
and lower lens eyes comprise 400 and 600 cone-like photoreceptors respectively. 
The light sensitivity is accomplished through a c type opsin (cnidops), cAMP, and 
an opsin Gs pathway which hyperpolarizes the photoreceptor membrane [31, 32]. 
The photoreceptors are interconnected through invaginated chemical synapses [33] 
and, since the box jellyfish retina is everted, articulate directly on putative second 
order neurons. The everted retina means that the light strikes the outer photoreceptive 
signals directly and that the animals do not have blind spots as in vertebrate eyes, 
where the afferent optic nerve exits the retina (optic disk). 


2.5 Rhopalial Nervous System Organisation 


The organization of the RNS is not yet understood in detail. However, visual infor- 
mation, detected by the lens eyes, is processed in putative second order neurons and 
subsequently applied to modulate swim pacemaker signal frequency. The swim pace- 
maker cell cluster is a central convergent and translatory hub for processed sensory 
information. The visual input in turn modulates the pacemaker cell cluster activity 
and is here translated into the motor signal that controls the motor effectors of the 
box jellyfish bell. The swim bell contractions are the only means of locomotion 
for the box jellyfish so the swim pacemaker signals can be used as indication of 
the behaviour of the animal. The advantage is, as mentioned, that we do not need 
to examine the entire animal to determine a behavioural reaction to a given visual 
stimulus [25, 34]. 


2.6 Retinal Organization, Bipolar Cells Connectivity to ON- 
and OFF- Ganglion Cells 


Vertebrate photoreceptors signal through a type c-opsin and a G-protein cGMP trans- 
duction cascade [35] and the photoreceptors thereby hyperpolarize in response to 
light ON stimuli [36]. In mammalian retinas, bipolar cells respond selectively to 
the photoreceptor activity. In the dark, the constant release of glutamate from the 
cone cells keep ON-bipolar cell membranes hyperpolarized, but at light-ON hyper- 
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polarization of the photoreceptor membrane reduces glutamate release, inducing 
depolarization of the bipolar cell membrane. In contrast OFF-bipolar cells remain 
hyperpolarized in light-ON conditions and depolarizes in response to light-OFF 
events. Retinal ganglion cells subsequently integrate neuronal information from sev- 
eral bipolar cells to produce firing rates corresponding to the received information. 
Ganglion cells have circular receptive fields with specialized center and antagonistic 
surround regions (eg. ON-center-OFF-surround). Depending on the type of ganglion 
cell the firing rate is modulated by light illumination of the center- or surround region. 
ON-center ganglion cells have low rates of firing under dim illumination and rapidly 
increase firing in response to light-ON stimulus in the center of their receptive field. 
In contrast, OFF-center ganglion cells discharge at low rates under light conditions 
and rapidly increase firing rates by light-OFF stimulus to their receptive field cen- 
ter [36]. The corresponding classes of bipolar and ganglion cells have excitatory 
connections so that an ON-center bipolar cell depolarization increases ON-center 
ganglion cell firing rate. The same is true for the OFF-center bipolar and OFF-center 
ganglions cell connection. There are species differences across the mammalian class 
but bipolar cells commonly integrate visual information from several light sensitive 
photoreceptors and, similar to the ganglion cells, have antagonistic center-surround 
receptive field organization [36]. 


2.7 Retinal Pre-processing of Basic Shapes (Bars, Contrast 
Lines, etc.) 


The firing rate of ganglion cells provides a measure of the difference in the intensi- 
ties of light illuminating the center and surround regions. Information about small 
differences in intensities and the center-surround organization of the ganglion cells 
receptive fields are therefore designed to report principally on contrast lines rather 
than absolute intensity [36]. Retinas adapted to optimally scan the appropriate envi- 
ronments and pattern recognition are inherent functions in visual systems across the 
animal kingdom [36, 37]. Unlike the human general-purpose eyes, that guide all 
our visual behavior, box jellyfish have special-purpose eyes, which scan the visual 
field for specific visual information (cf. matched filters) [37]. A visual strategy based 
on multiple special purpose eyes may seem energetically expensive, but requires 
far fewer computational circuits in the central nervous system. Based on immuno- 
histochemical staining, it is estimated that T. cystophora has approximately one 
thousand neurons available to process visual information from all six eyes on the 
rhopalium and to modulate several visual behaviors [17]. Accordingly, the rhopalial 
nervous system (RNS) is presumed highly compartmentalized, indicating that the 
actual neuronal network serving one particular eye is considerably smaller [24, 
30]. For the transfer into electronics, the pacemakers can be viewed as oscillators 
with a certain intrinsic frequency that is modulated by visual input. For Tripedalia 
cystophora it would be advantageous to have this type of matched filter in the lower 
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lens eye to scan the retina for contrast lines that would imply an oncoming underwa- 
ter obstacle. This could initiate evasive actions and steer the animal clear of collision. 
Presently, we have little knowledge of the cellular mechanisms involved in obstacle 
avoidance behavior but from previous work we can form an idea of how obstacle 
detection — or pattern recognition — can be accomplished in the box jellyfish visual 
system, and how the motor signal is generated. In the everted retina of T. cystophora 
the photoreceptors directly articulate on second order neurons, which could then be 
similar in function to the bipolar cells in the human retina. The second order neurons 
could translate the hyperpolarizing photoreceptor signal into ON- and OFF-center 
excitatory signals to the pacemaker cells. Three sets of giant neurons, presumably 
pacemaker cell clusters, have been identified in the T. cystophora RNS in the proxi- 
mal part of the stalk region. Each set has positive immunoreactivity (ir+) to a specific 
neuropeptide [18]. Since ir+ has been discovered for three specific neuropeptides in 
three distinct different pacemaker clusters and swim pacemaker signals (Fig. 3d) can 
be recorded from the efferent epidermal stalk nerve (Fig. 3c), it is attractive to suggest 
that two subordinate pacemaker cell clusters converge into a terminal pacemaker cell 
cluster, which produces the motor signal for the effectors. Additionally, it stands to 
reason that, due to general morphology of photoreceptor cells and the distance from 
the lens eyes to the putative pacemaker cell clusters, at least one level of interneu- 
rons is present between the two cell types. This again supports the presumption of 
bipolar cell organizational analogy. Assuming two major control units in the rhopal- 
ium; the upper and the lower lens eyes, second order neurons from each eye could 
converge onto one giant neurons/pacemaker cell cluster (sub-cluster) analogous to 
retinal ganglion cells in mammals. The sub clusters would converge onto the termi- 
nal pacemaker cell cluster that would integrate the signal from the entire RNS and 
produce the appropriate behavioral modulatory motor signal of the epidermal stalk 
nerve. The convergence of signaling from the two lens eyes and the bias of the two 
potential cluster outputs in the terminal cluster is beyond the scope of this chapter, 
but there is evidence that sensory input from the lower lens eye has priority over the 
other eyes on the rhopalium [25]. Theoretically, this bias could be integrated at the 
level of the sub-cluster—terminal cluster junction. 


2.8 Modeling RNS Visual Information Processing 
to the Mammalian Retina 


Here we will consider the modulation of obstacle avoidance behavior by the lower 
lens eye. Putatively this could be based on pattern recognition in the retina and 
oscillator frequency synchrony in the pacemaker cell clusters. Consider the pho- 
toreceptors of the lower lens eye, the second order neurons, and the corresponding 
pacemaker cell cluster in T. cystophora to be three levels of matrices analogues to 
the photoreceptors, bipolar cells, and ganglion cells of the mammalian retina. 
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Matrix level I: photoreceptors 


Approximately 600 photoreceptors make up the retina of the lower lens eye in the 
box jellyfish, each with an acceptance angle of about 20° [12]. As mentioned, pho- 
toreceptors with c type opsins hyperpolarize in response to a light-ON stimulus and 
assumedly this is the case for box jellyfish as well [31, 32]. A light-ON response 
implies that the photoreceptors only register a change in light intensity, whereas con- 
stant light causes adaptation and image fading or blindness [38]. Adaptation occurs 
in photoreceptors across the Metazoa regardless of opsin type, and animals must 
have strategies in place to counteract adaptation. In mammals, oculomotor generated 
fixational eye movements constantly refresh the retinal image to avoid immobil- 
ity blindness [39]. The box jellyfish utilizes the bell contractions as fixational eye 
movements and can in this way detect immobile objects in their visual field [24]. 


Matrix level II: second order neurons 


The photoreceptors articulate directly on second order neurons in the T. cystophora 
everted retina. Potentially, and similar to bipolar cells, OFF-center second order neu- 
rons respond to depolarized photoreceptors (light-OFF) by remaining in a depolarized 
state, and hyperpolarize in response to hyperpolarized photoreceptors (light-ON). 
Conversely, ON-center second order neurons depolarize in response to hyperpolar- 
ized photoreceptors (light-ON) and hyperpolarize with depolarized photoreceptors 
(light-OFF) [36]. In short, ON-center second order neurons signal in response to 
light-ON, OFF-center neurons signal in response to light-OFF. Applied to a matrix, 
we would now have an array of second order neurons activated when a suitable visual 
stimulus (light or dark) is applied to their respective field of view (acceptance angle). 
When in the second order neurons a ‘pre-programmed’ visual image fills the field 
of view of the lower lens eye, all second order neurons could potentially depolarize 
simultaneously. So, if T. cystophora would encounter a speed limit sign (Fig. 4), the 
image would dovetail the matrix of ON- and OFF-center second order neurons and 
entirely excitatory signals would be relayed to the respective ON- and OFF-center 
sub-cluster pacemaker cells. One important aspect of the pattern detection is that 
since the photoreceptors are binary in nature (either responding to a stimulus or not) 
the response threshold is important. If the threshold is too low, too much redundant 
information will be relayed to the processing circuitry—if the threshold is too high, 
important information could be lost. Figure 4c displays a threshold of sufficient sen- 
sitivity to detect a speed limit sign in a cluttered visual environment (numbers within 
a circle). The T. cystophora response to the speed limit sign is an approximation 
since their photoreceptors do not respond to red light, but this problem will not be 
encountered in a CCD or CMOS monochrome sensor. 


Matrix level III: pacemaker cells 


Hypothetically, and similar to bipolar and ganglion cell connectivity in the mam- 
malian retina, several second order neurons converge on the sub-cluster pacemaker 
cells. This neuronal circuit architecture seems logical in that there are consider- 
ably more neurons in the neuropil than there are pacemaker cells. This morphology 
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Fig. 4 Representation of the image quality perceived by the Tripadalia cystophora retina. a Speed 
limit sign 60 km/h. b The same image as in (a) monochrome (single opsin), and adjusted for the 
approximately 600 pixels/photoreceptors resolution in the lower lens eye of T. cystophora. c Same 
image as in (b) with adjusted response threshold 


suggests a matrix of i.e. ON-center second order neurons to converge on one ON- 
center sub-cluster pacemaker cell. Likewise the OFF-center second order neurons 
converge on OFF-center sub-cluster pacemaker cells. The ON- and OFF-center sec- 
ond order neurons form excitatory synapses with ON- and OFF-center sub-cluster 
pacemaker cells respectively. This results in sub-cluster pacemaker cells (oscillators) 
high frequency firing in response to excitatory presynaptic potentials from the second 
order neurons [36]. Following this line of reasoning when recognizing a pre-encoded 
pattern, such as the 60 km/h speed limit sign (Fig.4), under water, all sub-cluster 
pacemaker cells synchronize to high frequency firing, terminal pacemaker cells ini- 
tiates obstacle avoidance motor signal. In our automated driving assistant analogy, a 
potential vehicle would be slowed (or accelerated) to 60 km/h. 


3 An Engineered Box Jellyfish by an Ensemble 
of Pulsed-Coupled Oscillators 


As described previously, the box jellyfish’s visual system and rhopalial nervous 
system (RNS) can be viewed as an actuator coupled vertebrate-like retina. The level 
of visual information processing in the RNS is comparable to the pre-processing in 
a human retina in regard to pattern recognition, contrast line detection, and contrast 
estimation. The difference is that where human retinal information is sent to higher 
processing centers in the brain to actuate behavioral actions, in box jellyfish the motor 
effector signal is created directly within the RNS. In the present work, we modeled 
the described pattern recognition and oscillator synchrony to achieve the terminal 
pacemaker cluster output necessary for obstacle avoidance behavior. To simplify, we 
used the humblest pattern possible: one white pixel, one black. The light intensity 
of each pixel was monitored by a sensory oscillator, which would increase firing 
frequency when an internal criterion was met. In our experiment, the pattern of the 
two pixels could be varied in four ways: ON-OFF, ON-ON, OFF-ON, OFF-OFF. By 
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Fig. 5 Abstract functional diagram to mimic visual information processing in T. cystophora. The 
two input oscillators respond to the light intensity on the photodiode (sensor input). When both 
input oscillators are activated in response to the appropriate visual stimulus, the sync oscillator 
initiates an activation sequence of 2 Hz spikes. See also Fig. 10 


using one ON-center and one OFF-center sensory oscillator (sub-cluster pacemaker 
cell equivalents) to monitor the presented patterns, the pattern meeting the criteria 
of both oscillators simultaneously would be the ON-OFF combination. In Fig. 5 
this functionality is portrayed in an abstract view graph comprising two interacting 
input oscillators for the stimulus 1 (ON-oscillator) and stimulus 2 (OFF-oscillator). 
These oscillators were coupled and connected to a “Sync” oscillator unit (terminal 
cluster pacemaker cell equivalent). When the two sensory oscillators recognized 
the ON-OFF pattern simultaneously, the oscillators fired in phase and produced 
increased signal amplitude. The sync oscillator responded to the increased amplitude 
by commencing firing and producing the motor signal needed to initiate obstacle 
avoidance behavior. When the oscillators were uncoupled no increased amplitude 
could be observed, and the sync oscillator remained silent. The technical realization 
of this process is discussed below. 

This sensory oscillatory experiment is a proof of principle that the rhopalial ner- 
vous system can accomplish pattern recognition and respond with evasive action by 
utilizing interconnected sensory modulated sub-cluster pacemaker cells. The termi- 
nal cluster pacemaker cell firing activity is in turn modulated by the synchronized 
activity (and thereby signals amplitude) of the sub-cluster pacemaker cells. In order 
to partly mimic the complex functionalities of the box jellyfish neuronal network 
in electronic hardware, an ensemble of relaxation-type oscillators were applied and 
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Fig. 6 A mechanical (a) and an electronic version (b) of relaxation-type oscillators 


simulated by LTspice XVII (Linear Technology). Relaxation-type oscillators exhibit 
the leaky integrate and firing (LIF) mechanism of a neuron and therefore were applied 
to electronically mimic basic neuronal functions [40]. 

To explain the basic function of such oscillators a mechanical and an electrical 
version of relaxation-type oscillators are sketched in Fig.6a and b, respectively. 
The mechanical model (Fig.6a) is based on a seesaw comprising a mass on one 
wing (left) and a permanent charging (here water) on the right wing. In case the 
mass in the right container becomes larger than that on the left wing, the seesaw 
suddenly, seeps and the water container empties. The seesaw turns back to its original 
position and the procedure repeats again in a self-sustained way. The electronic 
version (Fig. 6b) consists of a constant voltage source Vo, a resistor R, a capacitor 
C and a glow discharge lamp. The capacitor is charged by the current J via the 
resistor. If the threshold voltage of the glow lamp is reached, the lamp discharges 
the capacitor, accompanied by a short visible light spark and the entire process starts 
again [41]. The period of the pulse are set by the resistor, the capacitor and the 
threshold voltage of the lamp. The similarity to the leaky-integrate and firing (LIF) 
model for neurons is obvious. In this work semiconductor based relaxation-type 
van der Pol oscillators (programmable unijunction transistors) were applied [42]. 
The programmable unijunction transistor (PUT) 2N6027 functions as an electrical 
switch [43]. The PUT is part of a small circuitry comprising passive devices. The 
basic circuit is sketched in Fig. 7a. The three terminals of a PUT are the gate G, the 
cathode C and the anode A. With a constant supply voltage V, a current flows through 
R1. This current charges the capacitor C1 and the voltage at the anode of the PUT 
increases. If it reaches the threshold voltage of the PUT, the PUT will switch into the 
conducting state. Hence, the capacitor gets discharged by a current flow from anode 
to cathode. At the resistor R4 this discharge can be measured in form of a voltage 
spike. If the voltage from anode to cathode is sufficiently high, a current can flow. 
The threshold voltage depends on the voltage divider consisting of the resistors R2 
and R3 and hence can be adjusted. The spiking frequency of the oscillator varies 
in dependence of its circuit parameters. The time constant t = R; * C4 as well as 
the voltage divider R2 and R3 define the pulse frequency fosc of the oscillator. The 
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Fig. 7 Electrical circuit of a relaxation-type van der Pol oscillator (a), and the corresponding 
voltage curves (b). V4 shows the charging curve of the capacitor C1 and the instant discharge when 
the threshold voltage of the PUT is reached. The voltage Vc shows the discharge spikes of the 
capacitor at the cathode of the PUT 


voltage traces at the anode terminal V4 and at the cathode terminal Vc are depicted 
qualitatively in Fig. 7b. Three of such PUT-based oscillators were coupled to mimic 
basal mechanisms of the box jellyfish‘s visual system. 

As coupling mechanism between the oscillators, a gate to gate coupling is used 
(Fig. 8). The voltage at the gate is constant when the threshold voltage of the PUT 
is not reached. When the PUT switches, a current can flow and the voltage breaks 
down for a short moment in form of the reversed voltage spike at the cathode. The 
capacitors in the coupling are used to filter low frequencies, especially the DC part 
of the voltage. In the moment of a spike the capacitors are equivalent to short circuits 
and a current can flow. This means that the voltage divider changes its value and 
it is more likely for the second oscillator to fire, too. Therefore the two oscillators 
are pulsed-coupled [44—47]. If the frequency difference between both oscillators is 
small enough a synchronization will occur. Synchronization can only be achieved 
with oscillators that are similar enough to each other in regards of their frequencies. If 
the difference is too high even with a strong coupling no synchronization can occur. 
In this work the coupled oscillator scheme as shown in Fig. 5 has between transferred 
to an electronic circuitry design and in the first stage simulated correspondently by 
LTspice XVII [8]. The pattern recognition functions with three oscillators and persists 
of two stages. The visual stimuli are encoded in the voltages V1 and V2. A high 
voltage is equal to an ON pattern and a voltage of OV equals an OFF pattern. Both 
voltage sources are chosen in a way that all possible combinations of two binary 
inputs (OFF-OFF, OFF-ON, ON-OFF, ON-ON) occur over time (Fig. 9). 

For the charging resistors R2 and R6 of the two oscillators the values 750 kQ and 
420 kQ) are chosen in a way that no synchronization can occur without the right stim- 
uli from V1 and V2. The 2N7002 are n-mosfets and function as voltage-controlled 
switches. If a positive voltage is applied by V 1 or V2, the corresponding n- MOSFET 
switches and a current can flow through the device. The oscillator influenced by V 1 is 
called the ON-oscillator. If a positive voltage is applied to the n-MOSFET, it switches 
and a second resistor is parallel connected to the charging resistor. The new value of 
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Fig. 8 Electrical circuit of 
two coupled relaxation-type 
van der Pol oscillators. The 
two oscillators are coupled 
via a RC circuit. The 
capacitors serve as a 
DC-de-coupling between the 
oscillators. A firing of one 
oscillator leads to a phase 
shift of the other oscillator 
and may lead to synchrony in 
case the coupling resistance 
is finite 


= Coupling 


Element 


the overall charging resistor can be calculated with Rees = Rı * R2/Rı + R2. The 
second resistor is chosen in a way that the overall resistance Rges is almost equal to 
the charging resistor of the OFF-oscillator. Therefore, in the moment of a stimulus 
synchronization of the two oscillators becomes possible. The second oscillator is 
called OFF-oscillator because synchronization is only possible in absence of a stim- 
ulus from V2. When a positive voltage is applied as stimulus by V2, the n- MOSFET 
switches into the on state and now a resistor is parallel connected to the capacitor. 
This leads to a leaky-current through the resistor and less current arrives at the capac- 
itor. Therefore, the charging process of the capacitor takes longer than before and the 
frequency of the oscillator drops. This drop is so high, that again no synchronization 
is possible between the two oscillators. The at the cathodes generated voltage spikes 
are the input for the second stage. The second stage consists of only one oscillator: 
It only spikes when the oscillators of the first stage are firing in synchrony. Two 
optocouplers (PC817A) are applied as voltage dependent switches and likewise to 
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Fig. 9 Electrical circuit of the complete setup. The ON- and OFF-oscillator are on the left and 
receive their input from V1 and V2. With the coupling element in place and the right input pattern 
synchronization can occur and the synchronization oscillator on the right starts to spike. Chosen 
Values: R1 = 1 MQ, R2 = 750 kQ, R3 = R7 = R17 = 100 kQ, R4 = R10 = R18 = 200 kQ, 
R5 = R8 = R12 = R13 = 1 kQ, R6 = 420 kQ, R9 = 1.6 MQ, R11 =2 MQ, R14 = R16 = 
10 kQ, R15 = 9 MQ, C1 = C2 = C5 = 0.1 uF, C3 = C4 = 1 uF, V3 = V4 = V5 = 10 V 


realize a potential decoupling from the input circuit. Both optocouplers are con- 
nected with a cathode of one of the PUTs. Hence, the generated voltage spikes of 
the oscillators are the inputs of the optocouplers. Only when both optocouplers are 
switched on simultaneously, a current charges the capacitor of the sync oscillator. 
Therefore, only when the optocouplers receive a voltage spike at the same time from 
their input-oscillators can the synchronization oscillator generate spikes—which is 
equivalent for synchronization of the input oscillators. 


4 Conclusion 


Biological model systems offer attractive guidelines to develop novel comput- 
ing architectures, which may exhibit enhanced pattern recognition capabilities, by 
equally low power dissipation. In this work, we demonstrate how two input oscil- 
lators and a sync oscillator can mimic basal visual information processing in the T. 
cystophora rhopalial nervous system. The sync oscillator fires only when the two 
input oscillators achieve the correct input pattern and subsequently initiates the acti- 
vation sequence (2 Hz spiking signal) (Fig. 10). This validates not only that pattern 
recognition functionality in the box jellyfish nervous system can be accomplished 
by sequentially connected oscillators, but also that this functionality can be reversed 
engineered into hardware electronics. The present work considers a very humble 
pattern of just two pixels but with a bit of engineering effort, hardware components 
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Fig. 10 The first two panels show the input variation over time (V1 and V2 in Fig.9). Panel 3 
shows the frequency of the ON and OFF oscillator in dependence of the input stimulus. Panel 4 
shows the spiking behavior of the synchronization oscillator that only spikes when the right input 
pattern occurs and induces a synchronization of the ON and OFF oscillator 


can be extended to support the approximately 600 pixels in the box jellyfish retinal 
sensor. This would enable extraction of specific visual information (such as speed 
limits) from a complex visual environment (Fig. 4). Pattern recognition filters for 
each possible speed limit inserted in parallel would then instantaneously report on 
the specific speed allowed, since only the sync oscillator recognizing the accurate 
visual cue would initiate the activation sequence. Subsequently, an autonomously 
driving vehicle would then be able to maintain correct speed using hardware units 
rather than occupying processing capacity on an on-board computer. Integrated in 
the T. cystophora model system is an override mechanism favoring sensory informa- 
tion of the most present danger. The special purpose lens eyes, upper and lower lens 
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eyes, each has pattern recognition function but for vastly different visual input. The 
upper lens eye is directed out of the water column and concerned about the keeping 
the contrast line between the mangrove canopy and the open sky in the periphery 
of the visual field, indicating that the animal is nested well within the habitat [15]. 
The lower lens eye is directing obstacle avoidance behavior and is responsible for 
keeping a safe distance to underwater obstacles that could damage the fragile bell 
of the animal [13]. Collision with obstacles present greater danger and avoidance is 
more time critical than to adjust the position of the animal in respect to the habitat. It 
is then logical that the lower lens eye has override privileges compared to the upper 
lens eye [25]. By adding pattern recognition filters with overriding privileges, of e.g. 
pedestrians or approaching vehicles, our hardware model would then have built in 
safety measures for clear and present danger — if the sync oscillators monitoring the 
override filters would initiate an activation sequence, the vehicle could be brought 
to an abrupt stop regardless of the input from the speed limit sync oscillators. The T. 
cystophora neuronal network integrates the bias of the sub cluster pacemaker cells 
into the terminal cluster pacemaker cells, but for the technical modeling we need 
another layer of sync oscillators to accomplish the decision making functionality 
(Fig. 11). 
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Fig.11 The T. cystophora nervous system contains an intrinsic terminal pacemaker override mech- 
anism to avoid immediate danger. For transference into electronic circuitry an extra decision making 
oscillator level must be added to accomplish this intrinsic override function. In this figure, “visual 
stimuli” represent the photoreceptor and second order neuron layers. The input layer comprises 
the sub-cluster pacemaker cells which respond to the appropriate visual input by high frequency 
firing. The sync oscillators in turn produce the 2 Hz activation sequence in response to aligned 
spiking from the sync oscillators. Here Sync2 has override privileges in case Sync1 and Sync2 are 
simultaneously activated, and an added decision making oscillatory circuit would evaluate this bias 
and produce the activation sequence appropriate for Sync2 
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Abstract Silicon neurons represent different levels of biological details and accu- 
racies as a trade-off between complexity and power consumption. With respect to 
this trade-off and high similarity to neuron behaviour models, relaxation-type oscil- 
lator circuits often yield a good compromise to emulate neurons. In this chapter, 
two exemplified relaxation-type silicon neurons are presented that emulate neural 
behaviour with energy consumption under the scale of nJ/spike. The first proposed 
fully CMOS relaxation SiN is based on mathematical Izhikevich model and can 
mimic a broad range of physiologically observable spike patterns. The results of 
kinds of biologically plausible output patterns and coupling process of two SiNs 
are presented in 0.35 4m CMOS technology. The second type is a novel ultra-low- 
frequency hybrid CMOS-memristive SiN based on relaxation oscillators and analog 
memristive devices. The hybrid SiN directly emulates neuron behaviour in the range 
of physiological spiking frequencies (less than 100 Hz). The relaxation oscillator is 
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implemented and fabricated in 0.13 ym CMOS technology. An autonomous neu- 
ronal synchronization process is demonstrated with two relaxation oscillators cou- 
pled by an analog memristive device in the measurement to emulate the synchronous 
behaviour between spiking neurons. 


Keywords Silicon neuron - Izhikevich model - Hybrid CMOS-menmristive - 
Integrate-and-fire - Ultra-low-frequency relaxation oscillator - Artificial synapse - 
Analog memristive device - Neuronal synchronization 


1 Introduction 


Living species are well adapted to their environments, a result of a hundred million 
years of evolution on earth. Due to constrains in space, time and energy, biological 
information processing in nervous systems of creatures are shaped during evolu- 
tion towards an optimum between capabilities and resource consumption [1-3]. It 
provides a benchmark for technical systems particularly when it comes to elaborate 
brain functions such as conscious awareness or decision-making which consume an 
incredible small amount of energy within a limited volume (i.e. a space of a brain) [4]. 
Although, there are continuous remarkable progresses in very large scale integration 
(VLSI) technology, the gap still remains between digital processors and biological 
computing systems [5, 6]. 

Neuromorphic engineering emerges and endeavors to develop intelligent machines 
with comparable biological computation and energy efficiency [6-12]. More recently, 
bio-inspired silicon neurons (SiNs) and artificial neural networks have been vastly 
investigated and developed to imitate the biological computing scheme [13—15]. The 
nervous system is a fine-grained parallel processing and highly linked neural network 
[5, 6, 16, 17]. Therefore, the large superiority of biological computing systems for 
certain tasks like sensory processing or pattern recognition comes from its real-time 
analog computation in which data processing (e.g., computing and learning) and data 
storage (i.e., memory) are inseparably linked. However, digital processors based on 
von Neumann architecture execute binary computing with a strict separation of data 
processing and storing [6, 18, 19]. Hence, a large part of power consumption is 
distributed on the data transfer between processing units and memory. As a conse- 
quence, the artificial neurons capable of real-time analog computing have recently 
received increasing interest in silicon neuron design [20]. Furthermore, due to high 
similarity to neuron behaviour models, the relaxation-type oscillators draw more 
attentions in recent researches [21, 22]. 

In this chapter, two exemplified high energy efficient relaxation-type silicon neu- 
rons are presented. These two analog silicon neurons represent two promising devel- 
opment strategies in artificial neuron design. One strategy is fully CMOS technology 
spiking neurons based on mathematical neuron models (here: the Izhikevich-model) 
and generating a broad range of physiologically observable spike patterns. Spiking 
frequency, i.e. operating speed, of these artificial neurons is considerably higher than 


Biologically Inspired and Energy-Efficient Neurons 359 


that of biological neurons (e.g. typical mean firing frequencies of biological neurons 
are in the range of up to 100 Hz). Another strategy is hybrid silicon neurons, which 
use CMOS analog oscillators and analog memristive devices to implement neuron 
and synapse circuits. These hybrid SiNs directly emulate the behaviour of neurons 
at the biological frequency. Both SiNs introduced here allow real-time analog com- 
putation rather than binary data processing and enable the realization of large scale 
monolithic analog neural network in the future. 

Before introducing details of two exemplified SiNs, the analog memristive device 
used in the hybrid SiNs is introduced below firstly. Then Izhikevich-model based 
low-power SiN is introduced in Sect. 2 including the theories, topology, simulation 
and measurement results. In Sect. 3 the hybrid SiN based on low frequency relaxation 
oscillator is described. In order to understand the synchronization process of memris- 
tive coupling, the design and measurement results of single oscillator is introduced at 
first. Subsequently, the autonomous synchronization processes of two pulse-coupled 
oscillators via a RC network and an analog memristive device are experimentally 
verified, respectively. Finally, a brief conclusion in Sect. 4 completes the chapter. 


1.1 Memristive Devices 


The mutual memristive coupling of two self-sustained relaxation oscillators has been 
successfully realized experimentally [22], through the use of an Ag-doped-TiO2_ ,- 
Al memristive device with digital switching behavior. Compared with Ag-doped- 
TiO,z_,-Al memristive devices, interface-based memristive devices like the double 
barrier memristive devices (DBMDs) [23] with analog switching behavior (1.e., a con- 
tinuous change in resistance) behave more similarly to synapses in nervous systems. 
Consequently, to come closer to emulate the process of synchronization in neurons, 
in Sect. 3.3.3 an interface-based device will be adopted as an artificial synapse in our 
experiments. A brief overview of DBMDs is provided in the following. Interested 
readers are referred to the literature for more details [23-30]. 

The schematic of a double barrier memristive device is shown in Fig. la. It consists 
of a Nb/NbO,/Al,03/Nb,O,/Au material stack [23, 28], in which Nb and Au are 
the bottom and top electrodes, respectively, while Al,O3 acts as tunnel barrier and 
the Nb,O,/Au interface forms a Schottky-like contact [23, 24, 29]. The devices are 
produced by DC magnetron sputtering of all materials on 100 mm wafer without 
breaking the vacuum and subsequent structuring by standard photolithography, lift- 
off and etching [23, 24]. Figure 1b shows a typical absolute value of the current 
density versus voltage (|J| — V) hysteresis curve of DBMDs [23]. A gradual resistive 
switching behavior is present [23] rather than abrupt resistance jumps observed by 
digital memristive devices [22]. In the |J| — V measurement, the voltage has been 
ramped from 0 to 3 V to set the device from its initial high resistance state (HRS) 
to a low resistance state (LRS). Afterwards, the voltage has been ramped down to 
-2 V and back to 0 V to reset the device. Voltage was applied to the top electrode 
while the bottom electrode was grounded. Using low voltages (e.g., 0.5 V) allows 
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Fig.1 A schematic structure a and current density versus voltage characteristics of a double barrier 
memristive device plotted as absolute value on a semi-logarithmic scale (b). b is reproduced from 
[23] (licensed under CC BY 4.0—https://creativecommons.org/licenses/by/4.0/) 


for non-destructive read-out of the device state. The diode-like characteristic with 
a high J-V non-linearity obtained by the Schottky barrier facilitates integration into 
passive crossbar arrays [27], since the current at negative bias voltage is negligible 
compared to positive bias. The devices can further be gradually switched by using 
voltage pulses with different amplitudes and widths [26]. Switching occurs for pulses 
with a width in the millisecond regime or beyond. The non-linear switching process 
is further crucial dependent on the voltage amplitude. Moreover, the memristive state 
relaxes towards HRS with time [23]. These effects are considered in the coupling 
experiments shown in Sect. 3.3.3. Due to the diode-like characteristics, the main 
interaction between two oscillators happens when a positive voltage is applied to 
the memristive device, i.e. higher output voltage from an oscillator connected to the 
top electrode and lower output voltage from an oscillator connected to the bottom 
electrode, as it is shown in Sect. 3.3.3. 


2 Izhikevich-Model Based Low-Power Neuron 


In 2003, Izhikevich presented a mathematical model [31] that describes the spik- 
ing and bursting behavior of cortical neurons. This model combines the biological 
plausibility of the Hodgkin-Huxley type [32] and the computational efficiency of the 
integrate-and-fire model [33]. It has been described, in [31], as a two-dimensional 
system: 


v = 0.040? + 5v + 140 —u + I a) 


u' = a(bv — u) (2) 
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Fig. 2 Block diagram of the 
Izhikevich-model based 
low-power neuron 
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where v’ and uw’ are the derivatives of v and u with respect to time, respectively: 
v represents the membrane voltage; and u is a membrane recovery variable that 
provides a negative feedback to the membrane voltage [31]. The variable J adds 
incoming synaptic currents to the system. All variables in (1) and (2) and also the 
parameters a and b are dimensionless. 

In [13], the Izhikevich model was implemented in a 0.35 jum CMOS technology, 
consisting of 14 transistors for a single neuron. This neuron’s output, which represents 
the membrane potential measured at the axon, is tunable by the five different inputs. 
Figure 2 shows the different inputs and one output of the neuron. The output Vmem 
stays at a resting potential, as long as no currents at the postsynaptic input syn exceed 
the threshold. By changing the inputs V+, Va and V;n, different biologically plausible 
spike patterns can be generated [13]. The neuron uses a silicon area of 70 x 40 pm? 
and has a power consumption of 8 pJ/spike. 


2.1 Topology 


The schematic of this neuron is shown in Fig.3. We adapted the circuit presented 
in [13] for the usage in a low-power analog neural network. As shown in Fig. 3, 
the current input J,,, is directly connected to the membrane voltage output. In a 
neural network built without additional current output, the input current of the first 
neuron will couple directly to the second neuron. In the biological neuron as in Fig. 2, 
however, the current input J,,, is separated from the axon voltage Vinem. Therefore, 
a dedicated current output node delivering the current Taxon is added to the circuit. 
This output can be taken to represent an action potential propagating towards the 
synapse. Furthermore, we modified the biasing circuit of the original circuit. Instead 
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Fig. 3 Schematic of the neuron circuit, divided in four parts: the membrane voltage (blue), the 
slow variable (green), the comparator (yellow) and the axon output (orange) 


of a voltage bias via Vpias, we employ current-biasing using a mirrored input current 
Ibias. This enables more sensitive bias control in an integrated circuit. 

The neuron circuit can be structured into four parts: a circuit representing the cell 
membrane and, specifically, the membrane voltage (Fig. 3, blue), the slow variable 
(Fig. 3, green), acomparator (Fig. 3, yellow), and the axon output (Fig. 3, orange). The 
membrane circuit integrates the J;,, current on the capacitor Crem . The compara- 
tor controls a transistor which discharges the membrane capacitance to the resting 
voltage between spikes. This voltage level is defined by the applied input voltage 
V.. The slow variable circuit represents Eq. (2), and slows the depolarization of the 
membrane voltage. The input voltage Vz controls the amount of charge stored on the 
capacitor C,,. 

A possible configuration for the neural network is presented in Fig. 4. The first 
layer neuron output current is connected to the synaptic inputs of the second layer. 
The coupling strength is set via the axon output current by programmable weighted 
current mirrors. Although the neurons are interconnected, the individual membrane 
voltages of each neuron can be monitored through the output Vmem. In doing so, the 
output of the neural network can be combined using the information of the different 
membrane voltages. 


2.2 Simulation Results 


The circuit shown in Fig.3 is implemented in a 0.35 um CMOS technology and 
simulated in the Cadence design environment. It is able to generate multiple biolog- 
ically plausible spike patterns when a stimulation current is applied to the input. By 
changing the control parameters presented in the previous section, the output of the 
neuron varies. 
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Fig. 4 Configuration of multiple neurons in a neural network 


2.2.1 Individual Izhikevich-Model Based Low-Power Neuron 


The circuit can either be supplied with the nominal operating voltage for this technol- 
ogy of 3 V or with reduced power consumption with 1.5 V. The different operating 
modes are capable of generating different output patterns. The simplest pattern— 
single spikes—is shown in Fig.5. The input current is integrated on the membrane 
capacitance until the membrane voltage reaches the threshold voltage and a spike is 
generated. After the spike, the slow variable keeps the circuit reset for a short time. 
This pattern is generated with a supply voltage of Vpp = 3 V, Va = Ve = 0, Vin = 
700 mV, Ibias = 1 pA, and Jy, = 200 nA. All voltages refer to Vss (—1.5 V/-750 
mV). Even though the spike frequency is considerably higher than that of a biological 
neuron, the spike shape is realistic. 

With Ve = 100 mV and J,,, = 1 pA, fast spikes with a reset offset of 100 mV 
(shown in Fig.6a) are generated. The larger the value of J,,,, the faster Crem 1s 
charged; thus, the membrane voltage reaches the threshold voltage of the comparator 
faster. The state trajectory presented in Fig. 6b shows that the circuit reaches a limit 
cycle. This happens very quickly compared to the circuit in [13]. Additionally, there 
is only very small jitter in this state. 

In the low-power configuration the supply voltage is reduced to 1.5 V, Ibias is 
set to 20 nA, and J;,, is set between 100 nA and 1 pA. For a fair comparison of 
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Fig. 5 Simulated transient waveform of the circuit’s spiking membrane voltage 
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different circuit implementations and spike patterns, the energy consumption per 
spike is frequently reported, referring to the spike as the smallest computational unit. 
Thus, the total current consumed by the circuit is integrated over the time of a single 
spike event and multiplied by the supply voltage. Under low-power operation, an 
energy consumption of 8—10 pJ per spike is simulated, which is in good agreement 
with what was achieved in [13]. 


2.2.2 Coupled Izhikevich-Model Based Low-Power Neuron 


Two neurons are connected as shown in Fig. 4 to form a simple network. The current 
output Taxon Of neuron | connects to the current input /,,,, of neuron 2. The neurons 
are both operated in their low-power configurations, and the variables are set to 
Ve =130 mV, Va = 0 V, Vin =540 mV, Isynı =300 nA, and J;,,2 =200 nA, where 
Isynı is applied to neuron 1 and J,,,2 to neuron 2. Thereby, both neurons generate 
the same spike pattern with different spike frequencies. The state of each neuron can 
be seen by assessing the different Vmem outputs. The membrane voltages of neurons 
1 and 2 are shown in Fig. 7a. After about 30 us, both neurons become locked and 
oscillate in synchrony. Figure 7b shows the change in amplitude, while Fig. 7c shows 
the change in the frequencies during the settling process. As expected, the initially 
faster neuron | remains unchanged, while neuron 2 changes its output to the same 
frequency as neuron 1. Thereby, the frequency first rises close to that of neuron 1, 
but then drops to 830 Hz before beginning to rise again. Additionally, the amplitude 
changes during the settling process. While the frequency of neuron 2 is slow— 
between 15 and 20 jts—its amplitude is high, as more energy can be stored in Crem. 
After both neurons settle to the same frequency, the amplitude of neuron 1 remains 
larger than that of neuron 2, as J,,, differs between the neurons. 


2.3 Measurement Results 


The circuit was implemented and fabricated in 0.35 um CMOS technology. Figure 8a 
shows a microphotograph of the die, where the rectangular shape indicates the loca- 
tion of the circuit. Figure 8b presents the related layout of this neuron, with an area 
of 33.4 x 55.9 um? 

For all measurements, the neuron was operated in low-power mode (supply volt- 
age 1.5 V). All variable inputs were generated externally, using a laptop-controlled 
multifunctional input/output card. The output waveforms measured during a constant 
Isyn input are presented in Fig. 9a—d. Figure 9a shows simple slow spikes (V; = Vz = 
0 V, Vin = 0.9 V, Ibias = 20 nA, Isyn = 200 nA). The frequency was about 900 Hz 
and, therefore, close to biological spike frequencies. Additionally, the spike shape 
was in good agreement with that of biological spikes. By changing the input volt- 
age to V. = 100 mV and setting V;;, = 0.96 V, the neuron generated fast spikes, as 
presented in Fig. 9b. The spikes were fired with a frequency of 30 kHz. Due to the 
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Fig. 7 a Vinem of the coupled neurons during the settling process. b Change in amplitude during 
the settling process. c Change in frequency during the settling process 
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Fig. 8 a Photograph of the fabricated die with marked dimensions of 33.4 x 55.9 1m. b Layout 
of the presented neuron 
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Fig. 9 Measurement of different output pattern during a constant current input: a Slow simple 
spikes with a frequency of 900 Hz. b Fast simple spikes with a frequency of 30 kHz. ¢ Periodical 
bursting behavior. d Aperiodic bursting behavior 


high frequency, the neuron was not able to fully discharge to V.. Figure 9c shows 
periodic bursting (Ve = 0.1 V, Va = 0.1 V, Vin = 0.85 V, and Isyn = 1 pA). The 
neuron generated three very fast spikes and then discharged to V, periodically. The 
opposite -aperiodic bursting- is presented in Fig. 9d. The setup was the same as that 
for periodic bursting, except that Vz was set to Vag. 
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3 Ultra-Low-Frequency Hybrid CMOS-Memristive Silicon 
Neuron 


Synchronization and memory of spiking neurons are vastly accepted and explained 
as underlying mechanisms of neuronal signal processing, such as recognition, per- 
ception and awareness, in brains of living creatures [17, 34—40]. In neuromorphic 
engineering, to emulate neuron activities, a variety of ways to transfer these biological 
mechanisms to electronic circuits are realized [10, 41-43]. Researches in recent years 
demonstrate that the neuromorphic system built by hybrid CMOS-memristive silicon 
neurons consisting of relaxation oscillators and memristive devices is a promising 
candidate for neuromorphic computing, since they allow one to emulate neuronal 
synchronization and synaptic functionalities in a detailed way with energy efficiency 
and a high packing density [11, 19, 22, 44-49]. One example is given in Fig. 10. It 
shows an experiment based on two discrete self-sustained relaxation oscillators cou- 
pled with an Ag-doped-Ti O2-x-Al memristive device to emulate basal coupling and 
an autonomous synchronization scheme for neuronal ensembles [22]. More specif- 
ically, two relaxation oscillators with intrinsic oscillation frequencies of 543 Hz 
and 414 Hz, respectively, are pulse-coupled through a resistor—-capacitor network 
comprising a single memristive device. A synchronization process is observed as 
anticipated, i.e. the left ‘slow’ oscillator eventually follows the right ‘fast’ oscillator. 

The experiment successfully demonstrates that, with the memristive pulse- 
coupling of two relaxation oscillators, two relevant dynamic aspects of biology 
(memory and synchronization) can be transferred to electronic circuits in a small- 
scale compact system. In the future, the construction of a large scale pulse-coupled 
memristive oscillator network is expected to facilitate the emulation of higher cogni- 
tive functions and perceptual processes. Very large-scale integrated (VLSI) circuits 


Oscillator 2 


f2y=543Hz 


Fig. 10 Block diagram of discrete setup of two mutually coupled self-sustained relaxation oscilla- 
tors with intrinsic frequencies of fıy = 414 Hz and foxy = 543 Hz, respectively. M(x,t) (middle) 
is a memristive resistor network, where x is the state variable of the memory process 
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have been shown as a feasible solution owing to their variety of advantages: highly 
complex neural network connections, highly efficient parallel computing, and real- 
time processing [7, 12, 19, 20, 50]. However, to directly transfer the design in the 
experiment given in Fig. 10 to neuromorphic VLSI network, the structure of the 
oscillator from the experiment has two constraints: at first, the oscillator is based on 
a programmable unijunction transistor (PUT; 2N2067), discrete resistors and capac- 
itors. It is necessary to find a new circuit structure to replace PUT by using basic 
elements (i.e. CMOS transistors) in VLSI; In addition, the frequency of oscilla- 
tion and static power consumption of the oscillator from the above experiment are 
limited by discrete resistors and capacitors. Especially for the biological frequency 
applications—at which the frequency should be less than one hundred hertz and 
below—it is infeasible to realize ultra-large resistors or capacitors in the custom 
on-chip design to satisfy the ultra-low frequency requirements. As a result, a new 
integrated relaxation-type oscillator is highly demanded. 

In this section, we present a study of a hybrid CMOS-memnistive silicon neu- 
ron consisting of a monolithic ultra low frequency relaxation oscillator which is 
designed and realized in CMOS technology and the analog memristive device intro- 
duced before. The circuit design, simulation and measurement results of the proposed 
relaxation oscillator are introduced at first. Then, the principle of pulse coupled oscil- 
lators is explained and examined by experiments of resistive coupling system. At the 
end, an autonomous synchronization process is demonstrated by experiments of 
memristive coupling system. This hybrid SiN is strongly biologically oriented and 
paves the way for large neuromorphic VLSI system. 


3.1 Ultra-Low-Frequency Relaxation Oscillator 


This section introduces the design, the oscillation mechanism, the simulation and 
measurement results of the relaxation-type oscillator which is fabricated with 130 
nm IHP technology. The supply voltage 3.3 V is adopted to meet the state transition 
requirements of the analog memristive devices used to realize a coupling between 
on-chip oscillators. To make oscillator self sustained, negative differential resistance 
(NDR) circuit is adopted here. 


3.1.1 Circuit Design and Oscillation Mechanism 


The architecture of the proposed integrated self-sustained relaxation oscillator [9] is 
shown in Fig. 11. To overcome the obstacle of integration of ultra-large resistors or 
capacitors induced from the structure of oscillator in previous experiment, it adopts a 
pA-scale charging current to realize ms-scale charging time and biological oscillation 
frequency. The output current of a MOS-only current reference Irgr (1 nA) [51] 
is mirrored through two pairs of NMOS current mirrors (N3 and N4, N3 and N5) 
and two pairs of PMOS current mirrors (P1 and P3, P2 and P4), in order to supply 
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CL 


V 


Fig. 11 Schematic of proposed oscillator, which is designed to represent the discrete oscillator in 
Fig. 10. The output voltage port Vz is an interface to outside 


the charging current J, (250 pA) and output stage current J, (3 nA), respectively. To 
improve the accuracies of the current mirrors, cascode current mirror structures are 
adopted (but not shown here). Furthermore, the output stage current J, flows into 
configurable diode-connected stacked NMOS transistors (N6, N7, N8, and N9) to 
generate the output voltage V,, which adjusts the threshold voltage V;ng of negative 
differential resistance (NDR) circuit [52]. It avoids the difficult tradeoff of size of 
resistors or power consumption in the structure designed by the experiment shown in 
Fig. 10. The NDR circuit consists of PMOS transistor P5, a pair of NMOS transistors 
N1 and N2, and the configurable diode-connected stacked NMOS transistors. 

The oscillation frequency is tunable by applying a digital configuration to the 
programmable capacitor Ce. It consists of binary weighted capacitance with the size 
of 1, 2, 4, 8 and 16 Co. By considering chip area and reducing parasitic effects, 
the unit capacitance Co is designed to be 0.94 pF with the size of 25 um x 25m. 
This enables a programming range of C. from 0.94 pF to 30 pF. In addition, the 
tunable transistor N2 can adjust the NDR region. Another configurable module is 
the diode-connected stacked NMOS transistors which control the output voltage Vg 
in the range of 2.6-2.8 V in order to ensure the state transition of the memristive 
device. 

A timing diagram for the voltages V,, V,, and V, under the conditions of f= 19.7 
Hz and C,= 4.7 pF is illustrated in Fig. 12. When the circuit is powered on, the 
potential of V, and V, is zero and P5 is turned off. The constant charging current [, 
from P3 begins to flow into the programmable capacitor C. and potential V, rises 
linearly to Vay. At the same time, the output stage current J, from P4 flows into 
the diode-connected stacked NMOS transistors and generates output voltage V,. To 
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Fig. 12 Timing diagram of oscillator output voltage V, and internal voltage potential V, and Ve; te 
indicates the time required for charging Ce, while tg is its discharge time (Ce = 4.7 pF, f = 19.7 Hz) 


make sure P5 keeps turned off before V, arrives at Vıp, output voltage V, should 
reach V, y earlier than V, reaches Vay. The time of V, from 0 to V, y depends on the 
output stage current Z, and Cz which is parasitic capacitance at V, port. Considering 
parasitic capacitance from I/O Pad and additional test point on PCB, to make sure 
V, arriving V,y earlier, J, (3nA) is chosen 12 times bigger than J, (250pA) here. 
When the voltage V, reaches the turn-on voltage V;ng (i.e., Vay in Fig. 12) of the 
NDR, the transistor P5 turns on, C, begins discharging and Ze current flows into N1. 
It generates potential V, and turns on N2. In the design, the size of N2 is large enough 
to swallow all the current from J, when its V,, equals V, with small drain-source 
voltage Vy; (i.e. V¿). The drain-source voltage of N2 is then immediately pulled down 
(i.e., output voltage V, is pulled down to be V,,;,). This speeds up the discharging of 
C. and generates spike signal V,. 

The capacitor Ce discharges until the voltage V, reaches Vız and the gate voltage 
V, of N1 and N2 also decreases, which reduces the drain current of N2. J, flows 
back to the diode-connected stacked NMOS transistors. Thus, V, becomes high and 
switches off P5. Then, the next charging phase starts. From simulated waveforms 
in Fig. 12, we can see that V, presents a saw-tooth-type signal, V, is a rectangular 
pulse-type signal, and V, is a neuron spiking-type signal. 


3.1.2 Negative Differential Resistance Regime 


The V,—J, DC characteristics of the proposed oscillator, with different sizes of N2, 
are shown in Fig. 13. The width of the NDR region changes with the size of N2; for 
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example, with an aspect ratio of 240 for N2, when V is less than V;zg, P5 is cut 
off. When V, = Ving, then P5 turns on, and the oscillator enters an NDR region in 
which the voltage V, decreases as Ie increases. 

The negative-resistance region extends until the valley point is reached, where the 
current is defined by Isar. Beyond the valley point, further increases in V, produce 
increases in 7e. This region is the so-called saturation region, which should be avoided 
in the circuit designed for oscillation-based applications. An appropriate size for N2 
is chosen, in order to make sure that the charging current J, should be greater than 
Ing, thus guaranteeing the turning on of the oscillator and also not exceeding Isar. 
For this purpose, to make sure 7, (i.e. 250pA) is in the NDR region, the W/L of N2 
should be less than 240. In the design, the W/L of N2 is decided 220 and a 4-bit 
transistor bank is added to the oscillator to calibrate the size of N2, considering the 
deviation of the process. The unit size of the transistor bank is W/L = lum/1 pm. 


3.2 Simulation and Measurement Results of Relaxation 
Oscillator 


As shown in Fig. 14, the total area of chip layout is 0.05546 mm?, including the 
current reference, the programmable capacitor bank, and the core of the oscillator. 
The simulated static power consumption of a single oscillator is 24 nW excluding the 
power consumption of the current reference, as the current reference will be shared 
with other oscillators in future neuromorphic networks. The power consumption 
of the current reference is 255 uW. Figure 15 summarizes the performance of the 
proposed oscillator for different configurations, based on post-layout simulations 
and measurement results. The energy per spike is defined as the power consumption 
of a single oscillator core (active power) integrated over the period of spiking. Post- 
layout simulations indicate that it varies from 0.8 nJ/spike to 7.12 nJ/spike, and the 
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Fig. 14 The photograph of die with core area of 0.05546 mm”. The three main parts of this work, 
i.e. current reference (top), oscillator core (middle) and programmable capacitor bank (bottom) are 
shown in it 
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Fig. 15 Post-layout simulation and measurements of oscillation frequency and calculated energy 
per spike versus programmable capacitance 
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Fig. 16 Measured current versus IC number 


oscillation frequency is tunable from 3.15 to 81.30 Hz by programming the on-chip 
capacitors. A total of 10 chips were bonded and tested. The output of the current 
reference was tested, as shown in Fig. 16. Seven chips were within the acceptable 
limits for low-frequency neuromorphic applications. Compared with the simulation 
results, the oscillation frequency of chip 9 in the test results was programmable from 
3.58 to 13.01 Hz. The oscillation frequency was relatively lower when the charging 
capacitor was small (from | to 15 Co), due to the parasitic capacitor at the input of 
oscillators from PCB and chip package being comparably large, with respect to the 
small charging capacitor Ce. The parasitic effects can be reduced in the integrated 
on-chip design for neuromorphic network applications. 


3.3 Experiments of Coupling Systems 


In a previous work the coupling of two relaxation-type oscillators built by discrete 
Programmable Unijunction Transistors (PUTs) and by a digital type of memrisitive 
device was investigated by Ignatov et al. In this section two types of coupling exper- 
iments between both on-chip oscillators were discussed. The first experiment uses a 
RC network as intermedia, which aims to check the functionality of synchronization 
of the two oscillators. In the second experiment, the two oscillators are coupled by 
an analog memristive device. Due to its memristive characteristic, the test allows 
us to further observe the autonomous transition process from unsynchronization to 
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Fig. 17 Coupling test with RC network 


synchronization. This transition phase mimic the synchronization process of neuron 
signals [45, 46]. 


3.3.1 Coupling System with RC Network 


The coupling system of two oscillators via a RC network is shown in Fig. 17. Two 
capacitors Cı = C2 within the coupling network forms the DC potential decou- 
pling between both oscillators. The coupling network which consists of Rc serials 
connected with Cı and C3 is a passive high-pass filter with a cut-off frequency fe 
determined by the following equation [22]: 


1 


fe = wR.C 


(3) 


In the uncoupled state (i.e. without RC network), the intrinsic frequency fiy of 
oscillator 1 differs from the intrinsic frequency fzy of oscillator 2. Hereby, fiy < fon 
and the frequency difference is Afy = fon — fin. 

After the coupling system with RC network is powered on and at fo, the gate 
voltage V,ı and gate voltage V, are charged to be V, y (the blue and green curve in 
the lower graph in Fig. 18). At this moment, there is no current flowing through RC 
network due to Vz; = V,2. The voltage V,; from oscillator 1 (green dotted curve in 
the upper graph in Fig. 18) and V,2 from oscillator 2 (blue dotted curve in the upper 
graph in Fig. 18) rise linearly towards V,y as the capacitors C,; and Ce2 are charged 
with constant charging currents J, and J,2, respectively, as shown in Fig. 18. Induced 
by the difference in the charging capacitor selected for C., and Ce2, Va2 reaches the 
turn-on voltage Vg ahead of V41, as Ce2 < Cei, under the condition of the same 
charging current (i.e., Ia = l2 = Ic,). At the same time, V,> falls at the moment 
tı ahead of V,;. Therefore, there is a low-resistance signal path existing from V2 
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Fig. 18 Waveform of 
coupled process in 
synchronous phase 


to ground and the decreasing of V,2 cause voltage difference between V,, and Vy. 
There is current flowing from V,; to V2 through RC network. When V,2 falls down 
to be Ver (Ver © 0) at tia, so Voi is 


Ve = kede (4) 


where, Irc is the current flowing through the coupled resistor at the moment of tia. 
I. comes mainly from the Jọ; and the discharging current from parasitic capacitance 
at the node of V,;. To ensure that oscillator 1 follows the faster oscillator 2 (i.e., 
that the two oscillators synchronize), Vaz; (the value of Va; at the moment of tia) 
should be a threshold voltage V,,, bigger than V,,; in order to turn on pmos transistor 
P5_1 in oscillator 1 and lead Ce; into the self discharging state. Then, Vg; is pulled 
down to be V,; due to the turn-on of P5_1 oscillator 1. After that, the two oscillators 
independently go into self-sustained charging state again, and the synchronization 
process is repeated. As a result, the two self-sustained oscillators synchronize with 
the same frequency of fry. 

Therefore, Re needed for synchronization of RC coupled two oscillators can be 
calculated as 


Vopt = Re- Irc < (Van = Von) (5) 
Therefore, 
Va mE Von 
Re < (Wart = Von) (6) 
Irc 
where V,#1 is determined by 
le: tia = Vani + Cet 
hence, 
I, -tia le 
Vagı = (7) 
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where, ta © T. From Le- Tin = I-/fin = Van Cei, where, Tiy is the intrinsic 
oscillation period of oscillator 1, we can have 


L 
Ca = >_>: (8) 
fin: Van 
Substituting Eq. 8 into Eq.7, we obtain 
Vani = Van: fin/ fon- (9) 


Therefore, 


(Van . fi. = Von) [Van ü (d = STi = Von] 
< fon = Son (10) 


Irc Irc 


Č = 


From Eq. 10, it can be easily concluded that, when the frequency ratio of the two 
oscillators fın /f2n gets smaller (i.e., A fy / fon is larger), R should be smaller, such 
that the two oscillators can synchronize. Therefore, we can obtain the following con- 
clusions: The coupling resistance is proportional to the frequency ratio and inversely 
proportional to the frequency difference under the same value of foy. 


3.3.2 Measurement Results of Resistive Coupling 


To verify the above conclusions, five experiments have been executed and the corre- 
sponding results are summarized in Table 1. When the frequency ratio (fin /fow) 
varies from 0.207 to 0.889, relative frequency difference (Afy/f2y) identically 
decreasing from 0.793 to 0.111, the maximal coupling resistance R. required for 
synchronization increases from 564 KQ to 251 MQ. The variations of R, matches 
our expectation, since Eq. 10 is derived by ignoring parasitic factors. 


3.3.3 Coupling System with Analog Memristive Device 


In this section, an analog memristive device was chosen as coupling element in 
the coupling system. Here, devices showing similar electrical characteristics as the 
DBMDs presented above are used. These devices incorporate HfO> instead of Nb, O). 
While a typical |J|-V curve is shown below, a detailed analysis of the device per- 
formance will be published elsewhere. The circuit demonstrates an autonomous 
phase-locking and frequency synchronization process due to resistance changes in 
an analog memristive device. 

To test the performance of the memristive coupling system, two oscillators were 
assembled with an analog memristive device in a single test PCB as shown in Fig. 19a. 
The structure of test PCB is depicted in Fig. 19b. Oscillator 1 with lower oscillation 
frequency fıy and oscillator 2 with higher oscillation frequency fy are connected 
to the top and bottom electrode (BE) of the memristive device, respectively, through 
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Table 1 Measurement results of maximal coupled resistor for synchronization 


Oscillators Charging Intrinsic Freq. Ratio | Freq. Diff. Coupling 

capacitor(Co) | frequency of (fin/fon) (Afn/f2n) — | resistor (MQ) 
oscillators(Hz) 

fl 32 2.8 0.207 0.793 0.564 

f2 2 13.5 

fl 16 4.5 0.333 0.667 0.599 

f2 2 13.5 

fl 9 6.4 0.475 0.526 0.701 

f2 2 13.5 

fl 32 2.8 0.622 0.378 11 

f2 29 4.5 

fl 20 4.0 0.889 0.111 251 

f2 29 4.5 


Oscillator 2 


Fig. 19 Test system for the memristive coupling 


series connected DC decoupled capacitor Cı and C2. The gate voltages (V,; and 
V2) of both oscillators were recorded using a PicoScope 3000 Series mixed-signal 
oscilloscope after buffered AD820 amplifiers with high input impedance low input 
bias current to reduce the loss of signal current. 

The layout of a5 x 5 mm? chip containing analog memristive device is shown 
in Fig. 20a [48]. It contains 25 sub-cells arranged in a 5x5 array. Except for two 
sub-cells (Tcells, gray boxes) containing test structures, the other 23 sub-cells can 
be used in the experiments. Six devices with area sizes increasing from 100 um? to 
2500 um? [numbered with “1” to “6” in Fig. 20b] are located in a single sub-cell 
[23, 48]. To connect the memristive devices with the two oscillators, the chip was 
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glued onto a JLCC 44 chip carrier shown in Fig. 20c and some devices with an area 
of 400 um? (size 3) are wire-bonded for experiments. A typical |J| — V hysteresis of 
the HfO,-based devices is given in Fig. 20d. The JLCC 44 chip carrier is then placed 
into a PLCC 44 socket on the test PCB to connect two oscillators, as shown in Fig. 19 
for experiments. 


3.3.4 Measurement Results of Memristive Coupling 


As discussed above, the bigger the frequency ratio (fi/ fon) is, the larger coupling 
resistance is needed for synchronization. Hence, the two oscillators can immediately 
synchronize without a small coupling resistance, when the frequency ratio (fin/fow) 
is big. The initial resistance value of the memristive device is small enough for their 
mutual coupling and synchronization. However, for small frequency ratios ( fin/ fon), 
the two oscillators require a small coupling resistance for synchronization. When 
the initial resistance value is not small enough, as long as the resistance of the 
memristive device decreases gradually to R. defined by the Eq. 10, the two oscillators 
can synchronize. 

A representative synchronization process of two oscillators coupled with a mem- 
ristive device is shown in Fig. 21. A device with area size of 400 um? numbered with 
“3” in Fig. 20b was used in this experiment. By considering the switch dynamic and 
retention characteristics of the devices, the intrinsic frequencies of the two oscilla- 
tors 1 and 2 were fın = 2.24 Hz and fən = 5.90 Hz, respectively. Initially, due to 
the high resistance of the memristive device, the two self-sustained oscillators can 
not synchronize (i.e. they are in the desynchronous state (DS)). In each discharging 
period of oscillator 2, the V,; was pulled down to be V,p1 = Rm - Im when V2 
reaches V,; (about 0 V), where Ry was the resistance of memristive device, Im 
was the current flowing through the memristive device during each discharging from 
oscillator 2. As a consequence, every discharge of the faster oscillator (in this case, 
oscillator 2) would trigger a discharge of V,ı to be V,pı. During this period, voltage 
Veni exerted on the top electrode of the memristive device and the bottom electrode 
of the memristive was Vz. The voltage difference happened at each discharge period 
of the faster oscillator and gradually changed the resistance value of the memristive 
device. As long as the resistance of coupled memristive device decreased less than Re 
calculated by Eq. 10 and then V,,; < Vagı — Von, oscillator 1 can follow the rhythm 
of oscillator 2. In this experiment, after about 125 s, the two oscillators synchronized 
due to the state transition of the memristive device from a high to low resistance 
state. 

In upper graph of Fig. 21, the transient frequency variations of oscillator 1 and 2 
are given. In the desynchronous state phase (DS phase), the frequency of oscillator 1 
was equal to its intrinsic frequency (fı = fin = 2.24 Hz whichis calculated by 0.3 V 
threshold voltage). For the first 66 s, the oscillator network remained desynchronous 
until the intermediate phase (I phase) was reached. The intermediate state is charac- 
terized by the fact that the frequency of oscillator 1 jumps between the frequency foy 
and fiy at irregular time intervals. The intermediate phase ended till synchronous 
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Fig. 20 a The layout of a5 x 5 mm? chip containing analog memristive device. b Layout of a 
sub-cell. c Chip glued on JLCC 44 chip carrier and wire-bonded. d Typical |J| — V hysteresis of 
the HfO2-based devices 
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Fig. 21 Synchronization process of memristive coupling: Upper graph: fı and f) were the transient 
frequency variations of both self-sustained oscillators. The continuous resistance change of the 
memristive device caused a desynchronous phase (DS phase), an intermediate phase (I phase), and 
a synchronous phase (S phase). Middle graph: Transient waveform of V; and Vg2 signal. Lower 
graph: Zoomed in transient waveforms of V,; and Vg9 signal in desynchronous phase (in yellow 
background) and synchronous state (in green background), respectively, corresponding time slots 
in the whole synchronization process are marked by the yellow bar and the green bar in the middle 
graph 


phase (S phase) was reached at 125 s when phase and frequency of both oscillators 
synchronized. In the synchronous state phase, each V,2 pulse of oscillator 2 triggered 
oscillator 1 to discharging fully and generated a full gate pulse (i.e. amplitude of V, 
is from Vy to ground). Oscillator 2 oscillated at its intrinsic frequency fy over the 
entire period of time. 


4 Conclusion 


In this work, two types of biologically plausible silicon neurons have been realised in 
integrated circuit technology. The first low-power spiking silicon neuron is inspired 
by mathematical Izhikevich model and built by relaxation oscillator implemented 
in 0.35 um CMOS technology. It achieves an energy consumption of 8~10 pJ per 
spike under low-power operation mode and synchronous processes of two coupled 
neurons is demonstrated in the simulation results. The measurement results show 
that it can mimic a broad range of physiologically observable spike patterns. This 
area and energy efficient fully CMOS silicon neuron could be used as a universal 
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neuron circuit integrated in large scale analogue VLSI systems. The second ultra- 
low-frequency hybrid SiN achieves the biological spiking frequencies below 100 
Hz with energy consumption in the range of 0.8—7.12 nJ/spike. To emulate a basic 
neural network, two integrated ultra-low-frequency relaxation oscillators coupled 
by an external analog memristive device were proposed. The autonomous neuronal 
synchronization processes of this basic neural network are presented and analysed. 
The realisation of biologically plausible oscillators in CMOS technology and an 
analog memristive device fabricated on silicon wafer pave the way towards large 
scale hybrid analog integrated neural network realisation in future. 
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1 Introduction 


Oscillator networks proved to be natural mathematical models for a variety of com- 
plex processes in various areas ranging from biology and social sciences to physics 
and modern technology [36]. Due to a broad spectrum of application domains, oscilla- 
tor networks as mathematical objects constitute a wide range of dynamical systems: 
from paradigmatic phase oscillators to multi-dimensional neuro-inspired models, 
chaotic systems, and infinite-dimensional chemical oscillators. This together with 
the possibility of complex dynamical interactions between oscillators and complex 
interconnection topologies make the proper analysis of such systems challenging 
and complicates the understanding of the emergence of collective behavior therein. 
One of the fundamental types of oscillator networks behavior is synchronization that 
is known to be essential for many purposes. For example, the synchronization of 
oscillatory signals between different brain regions supports the interaction between 
working and long-term memory [16]. The phenomena of partial synchronization and 
multi-clustering are of practical importance for neurophysiological systems [35] and 
distributed power generation [3, 6]. 

Motivated by the above discussion, this chapter focuses on three essentially dif- 
ferent types of oscillator networks and provides their synchronization analysis. Thus, 
sufficient conditions for the emergence of multi-cluster formations in Kuramoto net- 
works with dynamic coupling are provided in Sect.2. This is made by proposing 
an alternative characterization of multi-cluster behavior in terms of the existence of 
invariant manifolds for the corresponding error-system [14]. These manifolds are 
of a special topological structure and their dimensions coincide with the number of 
clusters in the network. The conditions for the existence and stability of the invariant 
manifolds interrelate the natural frequencies of oscillators, plasticity characteristics, 
and the interconnection topology of the network. The proofs are based on the per- 
turbation theory of invariant tori of dynamical systems [37, 38]. 

Section 3 addresses the problem of the output-feedback synchronization of chaotic 
behavior generated by the ensembles of Chua oscillators and synchronization of spa- 
tiotemporal patterns in the master-slave configuration of two Gray—Scott reaction- 
diffusion models. The latter one is a prototypic model of complex isothermal autocat- 
alytic reactions that is governed by the pair of coupled partial differential equations. 
Synchronization analysis of this infinite-dimensional model is carried out employ- 
ing late-lumping observer design techniques [12] thus relating the synchronization 
property with the convergence of the observer error dynamics. It is worth noting that 
every considered setup requires a distinct method for its formal analysis. The usage 
of the presented theorems is demonstrated in the numerical case studies. A short 
conclusion in Sect. 4 completes the chapter. 


Notation 


The following notation will be used throughout the chapter. Let N, R, R.o, Rso, 
and C denote the sets of natural, real, positive real, non-negative real, and complex 
numbers, respectively. For given n,m € N let R” and 7,,, denote the n-dimensional 
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Euclidean space and m-dimensional torus, respectively. The one-dimensional torus 
Tı is the one-sphere (circle S1). Let f : Tm —> R” be a function of the variable 
P = (P1, ---, Øm)! € Tm Which is continuous and 27 -periodic with respect to each 
Qs, s = l,m. Finally, C(Tm) denotes the space of all such functions f equipped 
with the norm | flo = maxgez,, || f(g) ||, where ||-|| denotes the Euclidean norm in 
R”, i.e., IEO? ZN Ifi (I2, | fi(@)| stands for the absolute value of the ith 
component of f evaluated at p. By C!(7") we denote the subspace of C (Tm) with 
every f € C'(7,,) having a continuous partial derivative with respect to each gs, 
s = I,m and | f|ı = max{| f lo, | jlo. ah | lo). For a given set V, |V| denotes 
the number of elements in V. Re A(A) denotes the set of real parts of all eigenvalues 
of square matrix A and any set B < 0 if and only if for any b € B it holds that b < 0. 


2 Multi-clustering in Networks of Phase Oscillators 
with Dynamic Coupling 


In this section, the phenomenon of multi-clustering in networks of phase oscillators 
with dynamic coupling is studied. To this end, a general nonlinear model is intro- 
duced and the multi-cluster behavior of the network is then characterized in terms of 
the existence and stability of the corresponding invariant toroidal manifolds. These 
manifolds have a particular topological structure and their dimension coincides with 
the number of clusters in the network. Finally, the proposed approach is applied to a 
Kuramoto network with adaptive coupling and sufficient conditions are derived for 
the emergence of multi-cluster behavior. These conditions interrelate the dynamic 
properties of oscillators, the plasticity parameters of the adaptive couplings, and 
the interconnection topology of the network. A numerical example concludes this 
section. 


2.1 General Nonlinear Model 


Let G = (V, &) be the directed graph representing the network of nonlinear oscil- 
lators, where V = {1,..., N} and E C V x VY represent the oscillators and their 
interconnection edges, respectively. Let A = [ajj]@,jyevxv be the adjacency matrix 
of G, where a;; = 1 if the edge (i, j) € &, and a;; = O when (i, j) ¢ &. Contrary 
to many existing results, we do not impose any connectivity assumptions on the 
interconnection graph. The dynamics of the network is given by 


6; = fi0;) + J aijkijgij Oi, 0)), ie, (la) 
jEV 
kij = Tij (kij, 0i, 9), (i, j) € 8, (1b) 
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where 6;(t) € Sı and k;;(t) € R denote the phase of the oscillator i € V and the 
coupling strength of the link (i, j) € & at time t € R>ọ, respectively. The function 
fi € C(F,) defines the intrinsic dynamics of the ith oscillator, i € V, and gj; € 
C(F2), (i, j) € E defines the influence of oscillator j on oscillator i depending 
on their phases. The dynamics of the coupling strength k;;, (i, j) € & is defined 
by the function T;; : R x T2 — R such that Ij; (k, -) € C(T2) for every fixed k € 
R. For example, if fi (s) = wi, gij ($1, 82) = sin(s2 — s1), and Tj; (k, 51, 52) = —k + 
cos(s2 — sı), then (4) is a network of Kuramoto oscillators with dynamic Hebbian- 
type coupling [5, 14, 18]. If T; = 0 for all (i, j) € &, then (1) reduces to (la) with 
constant coupling strengths k;; € R, (i, j) € 6, and it defines the network of phase 
oscillators with static coupling (see e.g., [25] and [26], for the synthesis of electrical 
circuits for both adaptive and non-adaptive Kuramoto models, respectively). 


Definition 1 Let P = {P1, ..., Pm} withm € N, 1 < m < |V| bea partition of Y, 
where U"_ | P; = V and P; NP; = ifi A jforalli, j e V. The network exhibits 
cluster synchronization when the oscillators can be partitioned so that the phases of 
the oscillators in each cluster evolve identically. 


Contrary to the notion of multi-clustering given in Definition 1, the term multi- 
clustering may also refer to the network’s behavior that is characterized by a partition 
of nodes into subsets so that the frequencies of oscillators in each subset coincide, see, 
e.g., [4, 5]. This type of behavior also admits an alternative characterization in terms 
of the existence of a suitable invariant low-dimensional manifold (see Remark 1 for 
details). 


2.2 Synchronization Invariant Manifolds 


In sequel, an alternative characterization of multi-clustering given in Definition 1 
will be provided in terms of the existence of synchronization invariant manifolds of 
the corresponding error-system. To this end, for every cluster P,, s = 1, m we pick 
an arbitrary oscillator i; € P, within the cluster, denote its phase by ps := 6;,, and 
collect all gs, s = 1, m into the vector of reference phases g = (,..., Pm)! E€ Tm. 
For every oscillatori € P, \ {is}, lete; = 0; — p, denote the relative phase difference 
within the cluster P;, s = 1, m. All relative phase differences e;, i € V \ USL {is} 
and all coupling strengths k;;, (i, j) € & are collected into the vectors e and k of 
dimensions |V| — m and |E], respectively. 

In many cases, following the introduced notation, system (1) can be rewritten in 
the form 


(o = Fi (9, e, k), (2a) 
é = Fo (ọ,e, k), (2b) 
k = F3(9, e, k), (2c) 
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with suitably defined functions F1, F2, F3 such that F; (-, e, k) E C(Tm), i € {1, 2, 3} 
for every fixed (e, k). For instance, such a transformation is always possible when 
the interaction between nodes are of a diffusive type, i.e., g;; (01, 02) = gi;(0; — 0i), 
Vij (kij, 9, 0;) = rj (kij, 0; — 0i). System (2) has the same number of equations as 
(1): Eq.(2a) governs the dynamics of a single selected node within every set P,, 
s = 1,m, Eq.(2b) governs the dynamics of the relative phase differences within 
every set P,, s = 1, m, and Eq. (2c) governs the dynamics of coupling strengths. 

We are now in position to introduce an alternative characterization of multi- 
clustering in terms of the existence of invariant manifolds of (2) of a special topo- 
logical structure. 


Definition 2 We say that system (1) admits multi-clustering defined by the partition 
P if (1) can be rewritten in form of (2) and there exists an m-dimensional invariant 
toroidal manifold for (2) given by 


M= {(e, k, p) € RIV" x REx Ty, : e =0, k = ulo), p E€Tm} G) 


for some u = (u1, ... , Uje) E C(Tm). 


The invariant manifold M corresponds to the oscillating behavior of coupling 
strengths k preserving zero phase e = 0 difference within clusters. Although the 
function u is periodic with respect to every g;, i = 1, m, the oscillations of the cou- 
pling strengths are not necessarily periodic, since they are generated by trajectories 
on the m-dimensional torus Fm, which can be, for example, quasi-periodic in time. 

The existence of the invariant manifold (3) does not automatically imply the 
emergence of multi-cluster behavior in (1). It only indicates the possibility of such 
behavior if the trajectory of the error-system (2) reaches the manifold. The invariance 
property guarantees that the multi-clustering will be maintained for all times since 
the moment when the trajectory of (2) reaches the manifold. The emergence of the 
multi-cluster behavior can be associated to the asymptotic stability of the manifold 
(3): If the manifold is asymptotically stable then any point from a vicinity of the 
manifold will eventually converge towards it. In Sect. 2.3, sufficient conditions for 
the existence and local asymptotic stability of invariant toroidal manifolds of type 
(3) will be derived for a particular subclass of networks (1), namely, for Kuramoto 
networks with adaptive coupling. The conditions will provide a trade-off between 
the natural frequencies of oscillators, plasticity parameters of adaptive couplings, 
and the interconnection topology of the network. 


Remark 1 In the spirit of Definition 2, the frequency multi-clustering can be also 
characterized in terms of the existence of an invariant toroidal manifold 


Mereq = f(e, k, p) € RVI" x Rel x Tm :e=d, k= u(9), QE Tm} 
for some u = (u1, .. . , uje) € C (Tm) and constant d € R!'Y!— , Constant phase dif- 


ference between any two oscillators within a cluster will guarantee the same fre- 
quency for all oscillators that belong to the same cluster. Further extensions and 
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generalizations of manifolds M and Mfreq are possible to cover many other types of 
collective behavior like, generalized multi-clustering or practical synchronization. 
These are, however, out of the scope of the current chapter. 


2.3 Application to Adaptive Kuramoto Networks 


Let G = (V, &) be the directed graph defined in Sect. 2.1 that represents the network 
of oscillators with the adjacency matrix A = [a;;]; ;7y- Additionally, it is assumed 
that the graph does not have self-loops, i.e., aj; = 0 for all i = 1, N. The dynamics 
of the network is given by [14] 


. N 
6; = wi + >D aij kij sin(0; = 0i), i= 1, N, 
j=l (4) 


ki; = —vkij + mijTO;-4), iF =1,N, 


where w; € R and 6;(t) € Tı denote the natural frequency and the phase of the ith 
oscillator. The dynamics of the coupling strength k;; (t) € R is defined by parameters 
Hij, VY E€ Ryo andT € CHT) with IT}; =deE Ryo. 

We are interested in establishing the relationships between the emergence of multi- 
cluster behavior in (4) (in the sense of Definition 1) and the dynamical properties of 
oscillators, the adaptive couplings, and the interconnection topology. To this end, for 
a given partition P, let 6;,, and Esus be the subsets of & that correspond to the intra- 
cluster links and inter-cluster links, respectively. The cardinalities of these sets cj, = 
\Gin| and Cour = |Gour| characterize the interconnection structure of G with respect 
to the partition P. Additionally, let wmin = min;_y7y |Wil and Wmax = Max; -Ty (Wil 
denote the minimal and maximal absolute value of the natural frequencies. 

Sufficient conditions for the existence and construction procedure of the invariant 
toroidal manifolds that correspond to the m-cluster behavior of the network (4) have 
been proposed in [14] for the case of identical plasticity parameters uj; = u € Rso. 
However, the mentioned result does not answer the question whether the constructed 
invariant manifold is (asymptotically) stable. Later, this question has been answered 
in [13] by proposing sufficient conditions for the asymptotic stability of the invariant 
manifold for the case when the plasticity parameters ju;; are different for the intra- 
cluster and inter-cluster links. Namely, let 4;; = (1 € Ryo if the link (i, j) connects 
nodes within some cluster P;, s = 1, m, and uij = u € Ryo otherwise. For conve- 
nience, system (4) with the chosen set of plasticity parameters will be denoted as 
È (A, u) from now on, where the first argument stands for the plasticity parameter 
Lij Of the adaptive link connecting nodes within the same cluster, and the second 
argument stands for the plasticity parameter j1;; of the adaptive link connecting nodes 
belonging to different clusters. 

The derivation of the error-system follows the steps introduced in Sect. 2.2. Let the 
partition P be given. For every cluster P,,s = 1, m pick an arbitrary oscillator i, € Ps 
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and denote its phase and natural frequency by ps := 6;, and Ws := wj,, respectively. 
For every oscillator i € P;, let e; = 0; — gs define the relative phase-error within a 
given cluster P,, s = 1, m. Then, E (ñ, u) can be rewritten in the following form: 


Qs = Ws +) a4, jKi, j sin ej + >» 5 i, ji, j sin(e; + Oy — Qs), s = 1, m, (5a) 
JEP; rs jEP, 


ĉi = Wi — Ws + > [aijkij sin(e; —ei)— Gi, jKi, j sin e;| 
JEP; 
F > > [aijkij sin(e; — ei + Pr — Qs) — ai, jki,; sin(e; + 9, — 0s) | 
r#s jEP, Vie P, \ {is}, s= l,m, (5b) 
kij = —ykij + uI (ej — ei + pr — Ys) ViEPs, Yj EP,, s Er,s,r = 1,m, 
(5c) 
ki; = —ykij + AT (e; — ei) Vi, j EPs, i~j,s=1,m. 
(5d) 


System (5) is a counterpart of system (2) for the case of Kuramoto networks with 
adaptive coupling and it has the same number of equations as system (4). Equa- 
tions (Sa) describe the dynamics of m arbitrarily selected oscillators (one from every 
cluster). Equations (5b) describe the error dynamics within each cluster. Equations 
(5c) describe the dynamics of the coupling strengths between nodes of different 
clusters. Finally, (Sd) describe the dynamics of the intra-cluster coupling strengths. 
Let g = (@),..., Pm)! E Tm and e = (ei, vey Cim, )T e R” be the vectors col- 
lecting all cluster phases g;, i = 1, m and all intra-cluster relative phase errors e;, 
i € P, \ {is}, s = 1, m, respectively. Similarly, all inter- and intra-cluster coupling 
strengths are collected into the vectors k!”“" e R° and k"t e Rê, respectively, 
and k = (kim/e" i kint ral yy Following Definition 2, the multi-cluster behavior in net- 
work È (jt, u) is possible if system (5) possesses an invariant toroidal manifold 


M= {(e, k, p) e RN™ x Ret x Tye =0,k =u(y), p € Tm} (6) 


for some u € C(7,,,). This invariant manifold corresponds to the oscillating behavior 
of the coupling strengths k preserving zero phase error e within clusters. In [14], it has 
been shown that the inter-cluster coupling strengths cannot converge to some constant 
value say d simultaneously guaranteeing the convergence of the phase errors to zero, 
i.e., {(e,k, yp) € RY" x Rote x FT, : e= 0, k =d, € Tm} is not an invariant 
set of (5) for any constant d € R“ , Hence, the oscillating behavior of the inter- 
cluster coupling strengths is necessary for the emergence of multi-cluster formations 
in (4). 

Following the steps of the proof of Theorem 3 from [14], the following result can 
be obtained. 


Theorem 1 (Adapted from [14], Theorem 3) Let the following conditions hold true 
for system X (jt, p) and a given partition P: 
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(Al)  foranys = l,m and for any i, j € P, it holds that w; = wj, 
(A2) for anys,r = l,m, s Ær there exist constants cs, € N such that for any 


i EP, 
) dij = Csr, 


JEP, 


(A3) given Cmax := Max > Cs, it holds that 


s=1,m rÆs 
Wmin — UY! SCmax >0 (7) 
and j 
Wmax + MY ôC, 
4L 8 Sou y ee max a max 224. (8) 
Y << Wmin — LY~'8Cmax 


sAr 


Then, system (5) has an invariant toroidal manifold M that corresponds to the m- 
cluster behavior of © (ñ, u) defined by the partition P. 


Proof The proof is based on the perturbation theory of invariant tori for nonlinear 
extensions of dynamical systems on torus. We refer the interested readers to the 
monographs [22, 38], which provide the fundamentals of the mathematical theory 
of multi-frequency oscillations and to the papers [11, 33, 37], which address the 
persistence of the Green—Samoilenko function of the invariant tori problem under 
the perturbations of the right-hand side of the corresponding ordinary differential 
equations. A relation of these concepts to the synchronization analysis of oscillator 
networks is discussed in [14]. 


Conditions (A1)-(A3) allow for the following interpretation: 


e (Al) requires the natural frequencies to be equal within every cluster. 

e (A2) requires that the number of incoming links to every node within a given cluster 
P, from a different cluster P,, r 4 s is the same. Condition (A2) restricts only the 
number of links and does not require any symmetry of the corresponding adjacency 
matrix. It is worth to highlight that the intra-cluster couplings are generally not 
required for the emergence of multi-cluster behavior in the network since (A2) 
restricts only the structure of the inter-cluster connections. 

e (A3) establishes the relations between the natural frequencies of the oscillators, 
plasticity parameters u, y, ô and the inter-cluster interconnection topology. The 
procedure for verifying this conditions will be demonstrated in Example 1. 


In order to derive sufficient conditions for the asymptotic stability of the manifold M, 
the following auxiliary notation is introduced : For a given cluster P,, s = 1, m, let 


Synchronization Phenomena in Oscillator Networks ... 393 


e n, be the number of elements in the set Ps; 
e Gs C G be a subgraph that correspond to the nodes from P, and intra-cluster 
connections, i.e., 
Gs ={(Ps,&s) : Es = Ps x Ps NG}; 


e A, be the adjacency matrix of Gs. 


To define the residual connectivity of G, with respect to the node i,, the nodes inside 
each cluster are enumerated according to the rule P, = {i],..., igs iria inh where 


i} = is, 1.e., the selected node i, has a sequential number k, in the cluster P,. Then, let 


e A, bean (ns — 1) x (ns — 1)-dimensional matrix constructed from A, by remov- 
ing its k,th row and column; 


e A, be the residual adjacency matrix w.r.t. the node ig, i.e., 


üii eea Mi Aia «+. Ais 
isi} isi, 1 fisik 41 isins 
Z Aici «++ Gigi’) Gisif |) ++ + Aisig, 
a 1 ks-1 ks +1 ns 

A, = A — (9) 

p= s . . A 5 

jis... Gigs | Gi js. «+. Ajis 
ist) Isley —1 “lslky+1 Ist, 


e D, be the degree matrix of As, i.e., the diagonal matrix with diagonal elements 
equal to the sum of all elements in the corresponding row of As, and Dọ be an 
(ns — 1) x (ns — 1)-dimensional matrix constructed from D, by removing its k,th 
row and column. 


Theorem 2 (Adapted from [13], Theorem 2) Let the network X(t, u) satisfy condi- 
tions (A1), (A2), and (A3) of Theorem 1, i.e., there exist the synchronization invariant 
toroidal manifold M that corresponds to the multi-cluster behavior of the network 
given by the partition P. If 


(44)  foreverys=1,m 
sign (0) Rea (A, = D; ) <0, (10) 


then there exist po < u such that for all v < mọ the invariant toroidal manifold that 
corresponds to the multi-clustering of X(jt, v) is locally asymptotically stable. 


The ’stability-condition’ (A4) depends on the value of the plasticity function T eval- 
uated at zero (sometimes called learning rule) and the eigenvalues of the matrix that 
is constructed of those elements of the adjacency matrix A, which correspond to the 
intra-cluster connections. This makes a crucial contrast to the ’existence-condition’ 
(A2) that restricts the inter-cluster connectivity of the interconnection graph G. 
The usage of both Theorems | and 2 is demonstrated in the following example: 
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Example 1 [13] 


Consider a network of N = 7 Kuramoto oscillators (4) with adjacency matrix 


010/0100 
001/0001 
1001000 
A=]|010/0100 
010/0010 
001/0001 
001/1000 


natural frequencies w = (0.5, 0.5, 0.5, V0.8, v0.8, v0.8, 0.8) ", plasticity param- 
eters y = 0.2, 4 = u = 0.001, Hebbian learning rule T (s) = cos(s), and the desired 
two-cluster partition P = {1, 2, 3} U {4, 5, 6, 7}. 

Condition (A1) is satisfied thanks to the choice of w. Every node in cluster P, has 
exactly one incoming link from the nodes of cluster P2, and vice versa. Therefore, 
the resulting interconnection topology satisfies the condition (A2) and the network 
satisfies (A3) with characteristics Cout = 7, Cmax = C12 = C21 = 1, 6 = 1. Indeed, 


Wmin — ear Stee: = 5 — ee = 0.495 > 0 


and 


—l1 
max 6 max 

A af Y TUY Cmax 9.9615 < 1. 

y 


T. — 
Wmin — LY 'SCmax 


s,r=l,m 


sAr 


All conditions of Theorem 1 are satisfied. Asymptotic stability of the two-cluster 
formation can be concluded from the condition (A4) of Theorem 2. For this purpose, 
the nodes 3 and 7 are chosen as kı and k for clusters Pı and P2, respectively. Since 
sign T (0) = sign cos(0) = 1 > 0, (A4) for the cluster P; reads 


1 


0 
oli), Te): aeea) 
oto 


~ -2 1 
Rei(Ay ~ Dy) = Rex ( 11) =R- i£) <0 


0 
Ai= 0 
i 


so that 


-1 


For the cluster P2, condition (A4) reads 
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thus 


—2 1 0 
~ =e _ | Re(—-1 i) 
ReA(A2 — D,) =ReA}—-1—-1 1 J = pee 


-1 0-1 


All conditions (A1)—(A4) of Theorems 1 and 2 are satisfied. Simulation results are 
presented in Figs. 1 and 2. In particular, the numerical simulations in Fig. 1 demon- 
strate the conclusions of Theorem 2 and show that starting from a vicinity of the 
invariant toroidal manifold, which corresponds to the desired multi-cluster behavior 
of oscillators, the coupling strengths and phase-errors of the considered Kuramoto 
network converge to this manifold. Figure 2 depicts two snapshots of the graph G at 
the beginning (t = 0) and at the end (t = 500) of simulation. 


0.04 0.015 
0.03 0.01 
0.02 
0.005 
0.01 
ki J 0 
ei 0 
0.005 
-0.01 
0.02 0.01 
-0.03 -0.015 
() 100 200 300 400 500 0 20 40 60 80 100 
t t 


Fig. 1 Left figure: Evolution of absolute values of the phase-errors e;,i = 1, N within clusters. All 


e;(t) E 0, i = 1, N that correspond to the asymptotic stability of the invariant toroidal manifold 
M and the emergence of two-cluster formation given by partition P. Right figure: Evolution of 
coupling strengths k;j. The intra-cluster coupling strengths converge to the constant value, and the 
inter-cluster couplings exhibit quasiperiodic oscillations. Due to the choice of rationally independent 
natural frequencies, these oscillations are not periodic in time 


Fig. 2 Snapshots of the graph G at the beginning (left figure) and at the end of simulation (right 
figure). Colors of the nodes represent their phases. In the right figure, the blue connections denote 
intra-cluster links whose coupling strengths converge to a constant value. Light-grey links corre- 
spond to the oscillating inter-cluster couplings (see also Fig. 1). Red and orange nodes in the right 
figure form to two different clusters 
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3 Synchronization of Complex Dynamics 


This section is devoted to the synchronization analysis of complex dynamics. In par- 
ticular, we address the problem of the synchronization of chaotic behavior generated 
by ensembles of Chua oscillators and the problem of the master-slave synchroniza- 
tion of spatiotemporal patterns generated by the Gray—Scott reaction-diffusion PDE 
model. Besides theoretical results on the synchronization, both problems are illus- 
trated in numerical simulations. 


3.1 Synchronization of Chaotic Behavior 


Synchronization of chaotic systems has been intensively studied during the last 
decades [2, 27, 31, 32] due to their numerous applications, e.g., in secure com- 
munication [1, 41, 48], laser physics [28], and biomedical engineering [40]. In this 
context, the Chua circuit appeared to be one of the most interesting objects of explo- 
ration since it exhibits extremely rich dynamical behavior and variety of bifurcation 
phenomena despite its structural simplicity. As pointed out in [46], the investigation 
of synchronizability of coupled Chua oscillators may help to understand complex 
dynamical phenomena arising in networks of chaotic systems of more general types. 

Synchronization of two linearly coupled Chua oscillators has been studied in [7— 
9, 44, 46, 51]. A graph-spectral approach for the synchronization of networks of 
resistively coupled nonlinear oscillators has been proposed in [47], and the upper 
bound on the coupling conductance required for synchronization has been obtained 
therein. In this chapter, we present sufficient conditions for the synchronization of the 
network of N € N Chua oscillators interconnected with the static nonlinear coupling 
via the first state coordinate only. Our conditions provide a trade-off between the 
interconnection topology of the network, properties of nonlinear coupling function, 
and parameters of the Chua circuits in order to achieve synchronization. In Sect. 3.1.1, 
the network under consideration is defined and the main problem of synchronization 
is formulated. Then, we propose sufficient conditions for the synchronization of 
N e€ N coupled Chua oscillators with static nonlinear coupling satisfying a so-called 
sector condition (see Eq.(15)), and provide a numerical example to illustrate the 
usage of the derived conditions. 


3.1.1 Chua Oscillator 


We consider the extended Chua circuit 


& = al—€ + & — bi) Hu, é (0) = &10 (la) 
E = $i — fo + $3, &2(0) = £20 (11b) 
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& = —Bi. — y&, §3(0) = &30 (11c) 
y=& (11d) 


with scalar piecewise linear function 
1 
d(s) =as+ ge alls +1/-—|s—1|] VseR 
and parameters a, B > 0, y > 0,a < b < 0, or ina matrix form as 


x= Px + bu f(x), x(0) = xo (12a) 
y=cx (12b) 


with the state x(t) = BIQ E(t) &3 Ole ER xo = [E10 E20 Ex" e R?, external 
input u(t) € R, which will be later used to interconnect Chua oscillators, and the 
matrix P and vectors f, b,c given by 


—aa 0 —a(é1) 1 
P=] 1-111], f@= 0 , b=c=|0 
0 —f -y 0 0 


Now, consider a network of N € N nodes described by a graph G = (V, &) with 
node set V and edge set &, so that |V| = N. Let the associated adjacency matrix 
be given by A = {a;;}i,;=1,..., With zero main diagonal. Then, applying the output 
feedback 


jasi 


N 
uj = -9 ajk — yj) i=1, N 
j=l 


with an arbitrary nonlinear locally Lipschitz continuous coupling function k : R > 
R, the dynamics of N € N coupled Chua oscillators can be written as 


N 
Xj = Axi — bY ajk; — yj) + fi), x:(0) = xio 
= (13) 
yi = cxi, 
i=1,...,N. For any given n € R* let x = (x1, ..., xy) : R —> R?” denote a 


solution to (13) satisfying initial condition x (0) = 7. The Lipschitz continuity of the 
right-hand side of (3) guarantees the existence and uniqueness of the solution for any 
initial value n € R^. 
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Associated to this network consider the relative synchronization errors with 
respect to the node 1 


ej = Xi — X], i=1,N. (14) 


The relative errors e;; between arbitrary nodes i and j can be expressed using the 
relative errors e; and ej 


Cij = Xj —Xj = Xj — xX, — (Xj —X1) = |j — ej. 


Accordingly, instead of analyzing NOD 


relative errors e;; between connected 


nodes, it is sufficient to consider the behavior of the N—1 errors e;, i = 2, N. 

The problem addressed in the sequel consists in providing sufficient conditions on 
the system parameters, the nonlinear coupling function k and the network topology 
which ensure the synchronization of N € N coupled Chua oscillators, i.e., the global 
convergence of the norms of errors e; to zero: 


lim lel =0, i=2,N. 
too 


3.1.2 Nonlinear Couplings Satisfying a Sector Condition 

We consider a special case of Lipschitz continuous odd coupling k : R > R satis- 
fying a sector condition: Let there exist two constants k1, k2 > 0 with k2 > ki > 0 
such that for all s € R: 


k(s)s 20, k(—s) = —k(s),  kils| < |k(s)| < kalsl. (15) 


Additionally, define the following auxiliary matrices 


0 |az| +++ laz]| 0 |a23 — ai3| --- [azn — an| 

lazx2] 0 `. lazy] |a32 — a12| 0 "+. [azn — ain| 
Ay = . $ 2 Ag = . . 
lay2| lanl 0 lan2 — 412| |an3 — ail +> 0 


with zero main diagonals, (N — 1) x (N — 1)-dimensional identity matrix Z, and 
K = diag{k2,...,«n}, where k; = ye aij denotes the degree of node i = 1, N. 
Sufficient conditions for the synchronization of the entire network can be formulated 
in terms of eigenvalues of a matrix M defined below that interrelates the parameters 
of oscillators, coupling function, and the interconnection topology. 
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Theorem 3 (Adapted from [15], Theorem 3) Let matrix 


M= E — alja) I — kiK + (ko — k) Ai +ko:A2 al | 
> I —uol 


be Hurwitz, where 


1+ 2 
m= -Re (y -y +A). 


Then, the synchronization errors (14) between the states of Chua oscillators (13) 
with nonlinear coupling k satisfying (15) converge exponentially to zero. 


Example 2 [15] 


Consider a complete graph hosting N = 20 identical Chua oscillators (13) in its 
nodes. The oscillators are connected via the nonlinear coupling 


k(s) = 3s + arctans forall s €R, (16) 


and the parameters of oscillators area = 15.61, 6 = 25.581, y = 0, a = —1.142, 
b = —0.715. These parameters correspond to the chaotic behavior of each oscillator 
with the double scroll attractor [34] (see Fig.3 (left)). The considered nonlinear 
coupling strength (16) satisfies the sector condition (15) with constants kı = 3 and 
ky = 4. For the chosen parameters of the network matrix M from Theorem 3 reads as 


—54.78341 15.611 111-70 
u=| 1 aall 0 al 


where @ denotes zero (N — 1) x (N — 1)-matrix, and 1 denotes (N — 1)- 
dimensional vector (1 l- 1)". The eigenvalues of M lie in the open left half- 
plane of the complex plane C so that Re(A(M)) € [—56.0643, —0.0748]. Hence, 
from Theorem3 we conclude that oscillators achieve synchronization. The state 
evolution of the oscillators is shown in Fig. 3 (right). 


3.2 Synchronization of Spatiotemporal Patterns 


3.2.1 Gray-Scott Reaction-Diffusion Model 


The Gray—Scott model [17, 21], that is a simple prototype for the models of com- 
plex isothermal autocatalytic reactions, is governed by the pair of coupled reaction- 
diffusion equations 
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€ 
2 
Hor 


Fig. 3 Left figure: Double scroll attractor for the chaotic Chua oscillator. Right figure: Time evo- 
lution of N = 20 coupled Chua oscillators from the Example 2 


ða = Daða — ab? +a (1 — a) (17a) 
a,b = Dpd?b + ab’ — (a + B)b (17b) 


with homogeneous Neumann (non-flux) boundary conditions 


d.a(t,0) = d-a(t,L)=0, t>0 (17c) 
a,b(t, 0) = a,b(t, L) =0, t>0 (17d) 


and initial conditions given by 


a(0, z) = a(z), 7zEQ (17e) 
b(0, z) = bo(z), zEQ, (17f) 


where a(t,z) € R>ọ and b(t, z) € R>o denote the concentrations of two chemi- 
cal species at time ¢ € [0, oo) and at position z € Q := (0, L), L > 0, parameters 
Da, Dp, œ, and B are positive scalars, initial conditions do, bo € L? (2). The space 
of square integrable functions f : Q > R”, n € N is denoted by L? (Q) and it is 
equipped with the norm || f || = Cfa | f (z)|2dz)2, where | - | denotes either the abso- 
lute value or the IR”-distance, depending on whether its argument is scalar or a vector. 

System (17) may exhibit a variety of irregular spatiotemporal patterns in response 
to finite-amplitude perturbations of the steady state (a, b) = (1,0) [10, 21] (see 
also Fig. 4). These include a diversity of complex dynamical regimes ranging from 
steady states and stationary periodic solutions to traveling waves, pulse splittings, and 
spatiotemporal chaotic behavior. Pattern formation capabilities and the underlying 
mechanisms of the Gray—Scott model in one- and two-dimensional spatial domains 
have been reported and analyzed by mathematical analysis methods [23, 24, 45], by 
computer simulations [29, 30, 42, 43], and by experiments [19, 20]. 

The problem of synchronization of spatiotemporal chaotic behavior generated by 
the Gray—Scott model have been addressed in [49, 50] by means of the impulsive 
control techniques. In particular, a class of pinning impulsive controllers has been 
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(a) Stationary pattern of the concentration a (b) Non-stationary pattern of the concentration a 
for a = 0.02951 and 8 = 0.058. for a = 0.02, B = 0.047. 


Fig. 4 Typical patterns generated by the Eq. (17) in the interval (0, 2.5) with diffusion coefficients 
Da = 2- 1074 and Dp = 10~*. Both patterns a and b emerge from the same initial conditions, but 
for different parameters a, £ 


designed to stabilize and synchronize the spatiotemporal chaotic behavior. The cur- 
rent chapter addresses the synchronization problem of spatiotemporal patterns in the 
master-slave configuration in which the master system is given by the Gray—Scott 
model (17), and the slave Gray—Scott system is coupled to the original system via 
a finite number of spatial locations. This coupling enters the slave system dynamics 
as the in-domain injections of the master system state variables at the respective 
locations (see Eq. (19) for details). Following the late-lumping approach that relies 
directly on the original PDE description, we examine the minimal number of coupling 
connections sufficient for the synchronization of the master-slave configuration and 
additionally analyze the results in numerical simulations. In the considered setup, 
the slave system can be seen as a special kind of the point-wise innovation (PWI) 
observer [39] for the master Gray—Scott model. The synchronization between the 
master and the slave systems will correspond to the convergence of the observation 
error of the PWI-estimator to zero and sufficient conditions for this convergence can 
be concluded using analysis techniques from [12]. 


3.2.2 Master-Slave Setup and Main Results 


Let the master system be given by (17) and be coupled to the slave system via m + 1 
spatial locations at positions z = ¢; € Q:= [0, L], i = 0,m,m € N so that ¢ọ = 0, 
Cm = L,and z; — zi-1 =: d; > Oforalli = 1,...,m. System (17) is then equipped 
with the outputs 


y; (t) = a(t, &), y?(t) =b(t,g), t>0,i=0,...,m. (18) 


A particular case of m = 1 with dı = L corresponds to the availability of the bound- 
ary coupling only, i.e., no in-domain coupling points. The slave system is defined as 
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a copy of (17) with the in-domain injection of the concentrations y?, y? at respective 
locations ¢;, i = 1,...,m: 


3,4 = Dad? — âb? +a (1 — â) (19a) 
3b = D,32b + âb? — (a + Bb (19b) 
âlt, i) =y E), t20, i=0,m (190) 
ba, =y, t20, i=0,m (19d) 
a(0O,z) =a(z), zEQ (19e) 
bO, z) = boz), zeg. (19f) 


The main result is given in the following theorem. 


Theorem 4 (Adapted from [12], Theorem 1) Let the initial conditions ag, bo € 
L?(Q) and, bo € L? (Q) for both master and slave systems satisfy 0 < ao(2), bo(z), 
ag(z), bo(z) < 1 forall z € Q. Then, there exist constants o; > 0, i = 1, m such that 
the states of the master and slave systems asymptotically synchronize in the Ly-sense, 
ie., 


tim a(t, ) -â€ | = lim Joc, 9- be, | = 0, 
t—> œ t—> 00 


provided that di < o;, i = 1, m. 


Proof A constructive proof is based on the observer design technique proposed 
in [12]. The resulting constants o; provide the largest distance between the spatial 
coupling locations that enable the global synchronization of two Gray—Scott models, 
i.e., synchronization for any the initial conditions satisfying Theorem 4. 


Example 3 


The master system (17) is considered in the domain Q = (0, 2.5) with parame- 
ters Da = 2 - 1074, Dp = 1 - 1074, œ = 0.02, 6 = 0.047 and the initial conditions 
(ao, bo) = (1,0) which are perturbed to (ao(z), bo(z)) = (0.5, 0.25) at locations 
z € (0.925, 1.05) and z € (1.925, 2.05) and to (ag(z), bo(z)) = (0.25, 0.75) at loca- 
tions z € (0.25, 0.375). The slave system (19) is a copy of (17) with 6 uniformly dis- 
tributed coupling locations. The initial conditions for the slave system are selected at 
the steady state (do, bo) = (1, 0). A comparison of the behavior of the master system 
and the slave one is given in Fig.5, and the corresponding synchronization errors 
are provided in Fig. 6. The norms of these errors converge to zero as t —> oo. Addi- 
tionally, the synchronization errors for the cases of 3, 6, 11, 51, and 126 uniformly 
distributed coupling locations are depicted in Fig.7. These numerical simulations 
demonstrate that the smaller gaps d; between the coupling locations lead to smaller 
observation errors and their faster convergence to 0. 
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Fig.5 A comparison of the evolution of concentrations b and b. Six equidistantly located coupling 
points are marked with red crosses (right figure) 
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Fig. 6 Synchronization errors in case of 6 equidistantly located coupling points 
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Fig. 7 Evolution of the sum of the L?-norms of synchronization errors depending on the number 
of equidistantly located coupling points for t € [0, 5000] (left figure) and t € [0, 150] (right figure). 
The slave system driven by 3 coupling points does not converge to the master system, whereas all 
other setups lead to the convergence of the synchronization error to 0 with the convergence rate 
increasing with the number of coupling points 
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4 Conclusion 


The chapter provides synchronization analysis of three essentially different types 
of oscillator networks, namely, (i) Kuramoto networks with adaptive coupling, (ii) 
networks of Chua oscillators connected via nonlinear static output-feedback term, 
and (iii) the master-slave configuration of two Gray—Scott reaction-diffusion mod- 
els. Sufficient conditions have been presented for the emergence of synchronization 
phenomena in these networks in terms of the dynamical properties of oscillators, 
characteristics of the couplings, and the interconnection topology of the networks. 
Theoretical results are supplemented with numerical case studies. 
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Abstract Neuromorphic engineering is concerned with the emulation of biological 
learning and memory processes in hardware. The use of memristive devices, i.e., non- 
volatile memory devices, has given this field a significant boost in the last decade. 
However, most of today’s efforts are aimed at the hardware implementation of artifi- 
cial intelligence computational methods, while the emulation of biological compu- 
tational methods is less pursued. In the latter, however, there is enormous potential 
for information technology. For this, however, network-dependent cognitive func- 
tionalities from biology must be identified and transferred to technical systems. In 
this chapter, we will show a possible approach. Using the hippocampus, which is the 
central structure of the mammalian brain responsible for learning new information, 
as an example, it is shown how elementary cognitive functions can be investigated 
by behavioral tests in humans and how their functionality can be broken down to the 
network dependent functionalities. Furthermore, it is shown how these functionalities 
can be technically reproduced in a memristive network model. 


Keywords Neuromorphic computing - Memristive devices - Learning - 
Information processing - Cognitive systems 


1 Introduction 


Neuromorphic engineering goes back to Carver Mead, who in the 1980s used the 
then new silicon technology to emulate biological circuits of nervous systems [1]. His 
work was motivated by the hope to better understand the functions of the brain and 
to partly reproduce complex processes in technical systems. In addition to Mead, 
the physicist and Nobel laureate Richard Feynman and the mathematician John 
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Hopfield were among the fathers of neuromorphic engineering. With the phrase 
on his blackboard at time of his death "what I cannot create I do not understand", 
Richard Feynman provided the motivation for generations of scientists in the field. In 
recent years, neuromorphic engineering has experienced an enormous technological 
interest, as it also enables applications for the field of artificial intelligence (AI) by 
providing hardware that can be better tailored to the needs of AI [2]. In this context, 
neuromorphic systems are now seen as hardware realizations of biologically-inspired 
computational architectures, and the field has expanded significantly, especially in 
the last decade [2, 5, 56]. 

Important components for the realization of biological computational architec- 
tures in hardware are memristive devices (also called memristor), which allow to 
emulate local biological learning paradigms in hardware, in particular to emulate 
synaptic plasticity [2-4]. Memristive devices are two-terminal electronic devices 
that change their resistance by applying electrical signals and retain the change even 
after the electrical signals are switched off [5]. Thus, this class of devices belongs 
to non-volatile memory devices and, as we will show, are ideal for use as artificial 
synapses in neuromorphic systems [6]. 

The change in coupling strength between neurons, i.e., the synaptic plasticity, 
is the essential building block for learning and memory processes in biological 
nervous systems and forms the cellular correlate of dynamic biological information 
processing [7], as we will consider in more detail in Sect. 2.1. Thus, the emulation of 
cellular learning forms by means of memristive devices is central to the development 
of neuromorphic systems and requires the emulation of critical neural information 
processes within memristive devices. However, this directly raises the question of 
what the requirements for the devices are and which functions are essential for the 
network-level processes of biological information processing and memory forma- 
tion. Therefore, it is indispensable to acknowledge the neuronal network architecture 
and learning rules to design suitable memristive devices. This poses, however, further 
questions to biology as to how information processes operate in our brain. Here, it 
is the global network level that allows higher forms of learning and determines our 
behavior and actions [64]. 

Even though scientists have been researching for centuries how our brain works 
and what the secrets of information processing are, the functioning of the brain is 
only incompletely understood and we are far from a true comprehension of biological 
information processing [64]. However, in neurobiology, significant progress has been 
made in the last decades in exploring information processing and specifically memory 
formation at the mesoscopic level of different networks in our brain [8]. The progress 
made in the field of neurobiology has not only provided a deeper understanding of the 
computational processes underlying cognitive functions but also revealed the struc- 
tural foundations of neural circuits and network architectures responsible for these 
processes. These developments offer an excellent opportunity to investigate the rela- 
tionship between the structure and function of cognitive processes, with a particular 
focus on the memory function, as a model for technological emulation and modeling. 
In this context, the hippocampus, which plays a critical role in memory formation 
(as described in Sect. 2.2), has emerged as an especially promising candidate for the 
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emulation of neurobiological learning pattern with memristive devices. In addition 
to a large number of anatomical findings, hippocampus-associated and network- 
dependent memory functions have been identified (e.g. spatial navigation, pattern 
completion and pattern separation) that can be attributed to distinct neuronal networks 
in the hippocampus [9]. This allows the investigation of learning and memory func- 
tions on a global level in the form of behavioral tests in humans, which is subject of 
Sect. 3. However, this poses the challenge of using suitable tests to study forms of 
learning and to obtain as detailed information as possible about the performance of 
the hippocampus [9]. Starting from those functional relationships of global network- 
based learning forms, we will show in Sect. 4 how these can be reproduced within 
neuromorphic networks based on memristive devices. We will address particularly 
the question which device properties are important to mimic synaptic plasticity for 
the emulation of network-based learning forms. 

Thus, in this book chapter, we want to address the original motivation of neuro- 
morphic engineering and show how biological paradigms of network-dependent 
learning forms can be adapted for emulation with memristive devices. Furthermore, 
we demonstrate how global functionalities of memory formation can be extracted 
and reconstructed in a way that they can be described by simple network structures. 
The aim is to show how a conceptional bridge that can be built from the multi- 
dimensional global network level to the cellular level. For this purpose, we follow 
the systematics shown in Fig. | and demonstrate how cellular learning forms can be 
emulated so that it can be transferred within a multidimensional network level, i.e., 
how one can move from the microstructure of synapses and neurons to a mesoscopic 
structure of only a few connected neurons to a macroscopic structure of a nervous 
system. 


2 Neurobiological Learning Principles 


2.1 Cellular Learning Paradigms 


Basis of learning and memory are activity-dependent changes of the connections 
between individual neurons. These changes are achieved by a temporary strength- 
ening or diminution of synaptic connections and are referred to as synaptic plas- 
ticity [7]. Synaptic plasticity describes an increase in the efficiency of a synaptic 
transmission through repeated or persistent activity of the connected input cells. 


Long-term potentiation (LTP) is the classic paradigm of synaptic plasticity and is 
regarded as the cellular basis for memory formation. This cellular phenomenon was 
first described by Bliss and Lømo in 1973 in the CA3! area of the hippocampus [10]. 
However, most experiments on LTP have been performed at the junction between 
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Fig. 1 Modelling of memristive learning behavior: emulation of cellular learning forms and their 
integration into a neuronal circuit architecture to mimic cognitive behavior. For this purpose, a 
bridge must be built from the cellular level to the multidimensional network level 


CA3 and CA1 areas of the hippocampus [65]. Thereby, a short, high-frequency stimu- 
lation of axons? connecting areas CA3 and CA1 can elicit a sustained enhancement of 
the excitatory postsynaptic response potential (EPSP) (see Fig. 2). The transmission 
of these potentials from CA3 to CA1 occurs via the neurotransmitter glutamate to 
voltage-dependent NMDA (N-methyl-D-aspartate) receptors, whereby the glutamate 
influx triggers the EPSP at the synapse between Schaffer collaterals and CA1 pyra- 
midal cells. This high-frequency stimulation triggering an EPSP leads to a persistent 
strengthening of CA1 synapses and thus producing a long-lasting increase in signal 
transmission between two neurons. This elicited synaptic plasticity makes a crucial 
contribution to memory formation regarding LTP acting as a surrogate of information 
storage in the central nervous system (CNS) [11]. In this process of input-dependent 
increased cellular excitability, the synaptic transmission efficiency is increased over 
hours to days via a functional amplification and weighting of synaptic connections. 
In contrast to LTP, a long-term depression (LTD), in the sense of a reduction in the 
strength of synaptic transmission, can be evoked by low-frequency stimulation. Four 
principal properties underlie LTP/LTD (see Fig. 2): (i) input specificity, i.e., LTP 
only occurs at this synaptic connection, (ii) associativity, i.e., simultaneous activity 
at a stronger synaptic connection enables LTP at an associatively linked weaker 
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Fig. 2 Properties of long-term potentiation (LTP) in CA1 of the hippocampus exemplified by 
a single pyramidal cell receiving strong and weak stimuli: a Cooperativity, b Associativity, and 
c Specificity. The postsynaptic response potential (EPSP: Excitatory Postsynaptic Potential) before 
after long-term potentiation is outlined on the right of each figure. Adapted from [7] 


synapse, (iii) cooperativity, i.e., LTP can be evoked by cooperative activity of input 
signals, and (iv) persistence, synaptic transmission is increased over hours to weeks. 

NMDA receptors associated with LTP are also thought to be closely linked to the 
formation of higher-order network activity i.e., hippocampal place cell representa- 
tions leading to cognitive maps, and theta rhythm [12, 13]. It has been shown that 
learning indeed directly induces LTP processes in CA1 [11]; also, an impediment to 
the maintenance of LTP results in a disruption of spatial memory of already stored 
information [14]. Importantly, these fundamental mechanisms of memory forma- 
tion in the CA1 region are also putatively disrupted in human neurological memory 
disorders, such as transient global amnesia and Alzheimer’s disease [15]. 

Another correlate of synaptic plasticity is spike-time-dependent plasticity 
(STDP) and paired-pulse facilitation (PPF). In these plastic processes, augmen- 
tation of synaptic transmission is achieved by coupling input signals within a critical 
time window of a few to several hundred milliseconds. A detailed description of 
STDP is given in Sect. 4. 


2.2 Network Dependent Learning Paradigms 


The previously discussed mechanisms of synaptic plasticity are the basis for memory 
and learning in the brain. However, memory and learning, i.e., memories and events 
within a temporal-local framework, require more complex network structures. The 
biological substrates of various types of memory can be assigned to different areas 
of the brain [16]. In particular, episodic memory content of our personal experiences 
is critically reliant on the hippocampus [17, 18]. An effective memory system (i.e. 
minimal interference and maximal capacity) must provide at least two cognitive 
functions: first, the rapid storage of experiences as individual events (thereby avoiding 
the ‘overwriting’ of similar information—‘catastrophic interference’), and second, 
the retrieval of those memories, when similar events are encountered [19]. 
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In this context, two functions of hippocampus, namely pattern separation and 
pattern completion are highlighted here, as they are essential cognitive processes 
for the encoding and retrieval of episodes [19, 20]. Theories regarding pattern sepa- 
ration and completion processes that derived from computational approaches have 
been consistently supported by studies in rodents (see 21 for a review). Recent data 
recorded in humans using modern imaging techniques show that pattern completion 
and separation play a critical role in human (and other mammalian) learning and are 
subject to aging or degenerative processes, such as in Alzheimer’s disease [21, 22]. 

A schematic representation of pattern separation and pattern completion is shown 
in Fig. 3a. Pattern completion allows incomplete representations to be ‘completed’ 
by previously stored representations. Pattern separation is the process of representing 
and storing similar representations uniquely and non-overlapping (orthogonalized). 
The biological significance of this information discrimination is that new information 
does not overwrite similar, previously stored information. Overwriting would lead 
to catastrophic interference and ultimately to no new learning. 

A schematic representation of the network topology of the individual hippocampal 
fields is shown in Fig. 3b. The hippocampal dentate gyrus (DG) and the CA3 field have 
been attributed the function of pattern separation and completion, respectively. Recur- 
rent axon collaterals serve as the basis of pattern completion, which in turn enable 
auto-association networks [23]. In detail, information is routed from the entorhinal 
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Fig. 3 Schematic representation of pattern separation and pattern completion in the hippocampus. 
Pattern separation provides a distinct representation of similar and overlapping information, 
whereas pattern completion unifies similar and overlapping representations. b Formalized circuit of 
hippocampal function. The input via the tractus perforans is the input pathway into the hippocampus. 
DG: Emergence and self-organization of patterns of EC mediated activity by means of a heteroas- 
sociative network (completion of one piece of information by presentation of information from 
another category). CA3: Excitatory recurrent connections (12,000 collaterals/CA3 cell) mediate 
autoassociative encoding and recollection of episodic memory content in the sense of completing a 
memory content upon partial presentation of information. In addition, direct projections from EC to 
CA3 (2600 connections/CA3 cell) provide presentation of known information. Schaffer collaterals 
encode and mediate associations between CA3 activity and CA1 activity (directly from EC). In 
addition, recurrent connections from CA3 to DG exist. CA1: direct input from EC forms new repre- 
sentations, which are compared with the predicted representations from CA3 (comparator function). 
Adapted from [21] 
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cortex (EC) directly to the subnetworks in the DG and CA3 via the perforant path. 
Additionally, these subnetworks receive further projections via their own recurrent 
collaterals. The network in CA3 additionally receives information about the mossy 
fibers from the granule cells of the DG. In particular, the distinct recurrent network 
in CA3 serves as an auto-associative network that enables the completion of previ- 
ously stored information against the background of the presentation of incomplete 
stimuli. The mossy pathway from the DG to CA3 serves to establish separate pattern 
representation in the context of new learning and to reduce interference, whereas 
the direct input from EC enables the retrieval or presentation of known information. 
Animal experimental data provide evidence that dentate gyrus networks are neces- 
sary for pattern separation, whereas CA3 networks are critical for pattern completion 
[21]. Finally, the CA1 compares new representations from EC with the predicted 
representations from CA3 and has therewith a comparator function (cf. Fig. 3b [39]). 

Theories based on computational models of pattern separation processing and 
experimental rodent studies that measure the behavioural outcome on the basis of 
hippocampal place cell remapping both corroborate that the DG/CA3 network is 
critically involved in pattern separation [24]. Studies in humans support this finding 
by means of fMRI investigations that measured the activity of hippocampal areas 
during behavioural paradigms that tax pattern separation [22, 25]. 

This network function is further characterized by the fact that the represented 
network circuits are arranged multilayered in three-dimensional space to cause an 
increase in computational capacity. The pattern separation function is favored by 
adhering to the principle of sparse connectivity from the DG to CA3 cells (46 inputs 
per C3 cell) [9]. Furthermore, LTP for augmentation or deaugmentation of synaptic 
transmission is required for the storage of information in these networks (see Fig. 3b). 
Also, these networks show a significant fault tolerance (graceful degradation), in the 
sense that errors due to lost components can be compensated by the distributed 
representation. 


3 Investigating Hippocampal Functions in Animals 
and Humans 


High demands are placed on our memory system in daily life due to constantly 
changing environmental conditions that require a continuous mnemonic processing. 
As shown in Sect. 2.2, this functionality is mapped in the hippocampus by the cogni- 
tive network-dependent functions of pattern completion and pattern separation. These 
functions reduce interference between memories and generalizations about similar 
events, thus contributing to memory formation [21]. The challenge, however, is the 
study of these functions and, in particular, the identification of network-dependent 
mechanisms with respect to memory performance, as sketched in Fig. 1. In particular, 
it is challenging to elucidate the role of hippocampal subfield processing in pattern 
separation and -completion in humans. 
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In this section, we will show to what extent this can be optimized by applying 
behavioural tests addressing the human non-semantic memory. Here, we will first 
discuss classic memory tests and especially the Mnemonic Similarity Task (MST, 
[27]). We further show how behaviour in a pattern separation task can be studied 
in two human hippocampal lesion models: first, the selective CA1 subfield lesions 
in amnesia and, second, preferential neurodegeneration in DG/CA3 subfields in a 
patient cohort with a rare inflammation of the brain (LGI1 encephalitis). These natural 
hippocampal lesion models help to examine and understand a causal relationship 
between anatomical structures and pattern separation performance. The development 
of an alternative to the MST using sensory stimuli instead of depictions of objects, 
the Visual Sensory Memory Task (VSMT), then concludes this section. 


3.1 Mnemonic Similarity Task 


Classic standard recognition tests feature only two types of stimuli, old and new 
stimuli (see, e.g., 26). These two types of stimuli are typically called “Repeats” (for 
the old stimuli) and “Foils” (for the new stimuli). Inspired by concepts from compu- 
tational neuroscience, new concepts of memory subfunctions came up, including 
pattern separation and completion. Behavioural pattern separation in humans is 
commonly measured by means of specific match-to-sample tasks that include a third 
type of stimulus, so-called “Lures” [21, 22]. Lures are stimuli that are similar to but 
not identical with old stimuli. In due course participants are now given three response 
possibilities, namely, “old”, “new”, and “similar”. 

An established test comprising lures is the Mnemonic Similarity Task (MST, [27], 
see Fig. 3.1). This memory test comprises an encoding phase of items of everyday 
objects and a retrieval phase (see Ref. [27] for examples). During encoding, the 
participant is asked to classify these stimuli as either indoor or outdoor objects. At 
that point in time, the participant is not aware that they will have to remember the 
stimuli at a later point in time. During recall, the participant is presented with old, 
new, and similar stimuli and given the corresponding three response possibilities. 
Supposedly, those similar lures tax hippocampal pattern separation so that correctly 
identifying lures as similar suggests successful pattern separation abilities, whereas 
confusing similar lures with their corresponding targets would indicate a bias towards 
pattern completion. The advantage of the MST is the usage of everyday objects that 
allows the application in patients with neurological disease and aging participants. 


3.2 Human Hippocampal Lesion Models 


Memory impairment is commonly caused by an impairment of hippocampal func- 
tions due to neurological disorders or aging [28, 29]. In the following section, the 
aims are to show the mechanistic contribution of the human hippocampus to pattern 
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separation and to demonstrate the neurobiological processes within the hippocampus 
during consolidation of mnemonic information. Consolidation refers to the (time- 
dependent) stabilization of a memory after initial acquisition (encoding) into a long- 
lasting form. To study these subjects, we investigated natural hippocampal lesion 
models in memory impaired patients with selective hippocampal damage. 

Although regional neural activity of the hippocampus can be tested using func- 
tional MRI (see Ref. 25 for an example), those studies lack information regarding 
mechanistic aspects of causality about the subfield-specific computational processes 
and the causal role of hippocampal structure and its function. Therefore, we examined 
two hippocampal lesion models, where specific hippocampal subfields are impaired 
due to neurological diseases, as shown in Fig. 5. 

In Study I [38], the transient global amnesia (TGA) served as a model for a 
selective disruption of hippocampal CA1 neurons. A TGA is characterized by an 
abrupt cognitive deficit limited to a anterograde amnesia in the acute phase that 
resolves within 24 h. Typically, focal lesions restricted to area CA1 can be detected 
in MR-Imaging [30, 31]. Hence, it was suggested that a selective impairment of CA1 
during TGA causes a deficit in pattern separation. 

Study II [43] aimed at further elucidating the causal role of hippocampal subfield 
contributions to pattern separation. Here, an extremely rare patient cohort was 
studied. Patients with an inflammatory brain disorder positive for LGI1 antibodies 
were examined. These patients develop limbic (-hippocampal) encephalitis with 
persisting memory deficits (so-called LGI1 encephalitis; 32, 33) and have structural 
damage to the hippocampal system [34, 35]. The aim of this study was to investigate 
the pattern separation performance from DG and CA3 that are predominantly affected 
by neuroinflammatory changes due to LGI1 encephalitis [36, 37]. The hypothesis 
suggested that inflammatory lesions within the DG and CA3 subfields correlate to 
hippocampal pattern separation. 


The role of the hippocampal CA1 networks in pattern separation and recogni- 
tion memory. 


Here, we investigated the critical relay function of CA1 neurons in pattern separation 
performance using TGA as natural lesion model of a CA1 deficit [38]. Information 
processing within CA1 is characterized by the comparison of dual afferent projec- 
tions—from EC via the perforant path and from CA3 via the Schaffer collaterals 
[19, 23, 39]. The integration of those two projections within CA1 is assumed to 
facilitate an immediate retrieval and consolidation in neocortical long-term stores 
[9]. With regard to the contribution of CA1 neurons to pattern separation processes 
in humans, our results complement the current concept developed in computational 
models and experimental animal models [40]. Pattern separation function is relayed 
and facilitated by the DG that is in turn assumed to decorrelate overlapping memo- 
ries by sparse coding of neural signals from EC to CA3 [9]. For the transfer of 
mnemonic information to extra-hippocampal areas, CA3 projects to area CA1, the 
main hippocampal output area [39, 41]. The selective CA1 dysfunction caused an 
impairment in the transmission of the then-separated information from the DG/CA3 
network to the neocortex resulting in ineffective pattern separation performance on 
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the behavioral level [38]. This suggests that CA1 does not perform pattern separation 
on the neural level per se, but relays memory information from DG/CA3 networks to 
neocortical areas. Complementing the results of our studies, this assumption applies 
to the result of the dependence of pattern separation performance on hippocampal 
DG volume, but a weaker association to the volume of CA1 [38, 43]. The role 
of recognition memory by the CA1 volume can be conceptualized by the central 
position of CA1 as the functional readout of hippocampal circuit projections. By 
measuring hippocampal volume in LGI1 encephalitis, we showed that the volume of 
CA1 was the best predictor of recognition memory [43]. The integration of the dual 
afferent projections from EC and CA3 facilitates the restoration of a memory trace 
and thus recognition of an environmental cue [42]. Together, these results corroborate 
the view that CA1 is involved in both pattern separation and recognition memory 
processes. The functional readout of the hippocampal circuit to neocortical areas 
involved in hippocampus-dependent memory formation is thus highly dependent on 
the dynamics within the subnetworks. 


Dentate Gyrus Networks in Pattern Separation 


Theoretical models state that the DG performs pattern separation by the transforma- 
tion of overlapping input patterns into distinct, non-overlapping representations [17, 
20]. Hanert et al. [43] showed evidence that the DG volume was the best predictor of 
behavioral pattern separation compared to the volume of regions CA2/3 and CA1. 
This functional model of the DG in pattern separation processes has been confirmed 
by electrophysiological recordings in rodents [24]. Evidence for a separation-like 
activity within the DG in humans has been provided by high-resolution fMRI during 
a mnemonic similarity recognition paradigm [25]. These results complement our 
findings by presenting a structure—function relationship between pattern separation 
and the DG. These findings are also in accordance with previous studies that demon- 
strated a greater volume of the DG to be associated with a better discrimination of 
overlapping items [44, 45]. 

In summary, our results in neurological patients clarified the role of the human 
hippocampus and its specific subfield contributions to pattern separation and memory 
consolidation. It was found that the hippocampal DG as well as intact CA1 neurons 
are essential for pattern separation in humans. We also demonstrated that pattern 
separation was best predicted by the volume of the DG, whereas recognition memory 
was stronger associated with the volume of CA1. However, we also found that an 
impairment due to a lesion restricted to CA 1 neurons compromised pattern separation 
performance. These results emphasize and refine the current view on hippocampus- 
dependent memory processing within the hippocampal DG as a critical ‘pattern 
separator’, and CA1 essentially involved in transferring the mnemonic output to 
neocortical long-term stores. 
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3.3. The Visual Sensory Memory Task (VSMT) 


The hippocampus plays an important role when information is transferred from short- 
term to long-term memory. In principle, that could be any sort of information. Here we 
will focus on the difference between categorical-semantic and acategorical-sensory, 
non-semantic information. 

During information processing, the original sensory information is possibly 
enriched with semantic content: The letters of the words of a poem are initially 
nothing but a pattern of black and white areas. This sensory black-and-white pattern 
is processed to be interpreted as letters, the letters form words, and the words transfer 
meaning. Likewise, the sensory colored patterns of the stimuli of the MST (see Fig. 4.) 
gain meaning as they are processed, interpreted, and recognized. The four-leaf clover, 
for instance, could be seen as a sign of good luck by some persons. Semantic content 
is organized in hierarchical categories: A four-leaf clover is part of the category 
clover, which is part of the category plants. 

In the presence of such semantic content, the original sensory form of the input 
loses its importance; memorizing this information is based more on the higher repre- 
sentations rather than on the lower ones [46]. The exact amount and content of the 
meaning associated with such semantic stimuli might, however, vary enormously 
between persons; a florist might associate quite different things when seeing a picture 
of a four-leaf clover than a rabbit breeder. This introduces a massive variation of 
retention performance which is not under the control of the experimenter. 

Here we opted for non-semantic sensory information. This type of information 
shows behavioral characteristics identical to those known for semantic information 
[47]. It can thus serve as an example for the type of information typically treated by the 


Encoding Phase Test Phase 
Indoor/Outdoor? Old/Similar/New? 


Fig. 4 A schematic presentation of the Mnemonic Similarity Task. The four-leaf clover is present 
in both the encoding and the testing phase; it is an example of a “repeat” stimulus. The oilcan is not 
present in the depicted encoding stimuli; it might be a new stimulus, a “foil”. The seahorse shown 
in the test phase is similar to but not identical with the seahorse shown in the encoding phase; it 
represents a “lure” stimulus. The same holds true for the picnic basket 
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Fig. 5 Human hippocampal lesion models. a—c Study 1. a T2-weighted and diffusion-weighted 
magnetic resonance imaging (DWI) show representative lesions confined to the CA1 area of the 
hippocampus in TGA patients. b Overview of all DWI/T2 lesions of TGA patients participating 
in Study 1 transferred to an anatomical template of the cornu ammonis demonstrating selective 
CA1 lesions [38]. c TGA patients showed a strongly impaired pattern separation performance 
measured by the MST regarding all lure similarity levels (black). After recovery from TGA, the 
same patients’ performance (grey) was equal to healthy control participants (white). d-f Study 2 
[43]. d Tl-weighted magnetic resonance imaging shows hippocampal subfield segmentation in a 
representative healthy control participant compared with a patient with volume loss in DG and CA3 
regions due to LGI1 encephalitis. e) Higher volume of hippocampal DG predicts better pattern 
separation performance measured by the MST in patients (full circle) versus healthy controls. f) 
Patients with reduced hippocampal DG and CA3 volume show significant gradual increases in 
pattern separation performance from high to low similarity. However, the healthy control group 
exhibited significantly better performance in pattern separation from highly similar lures to lures 
with low similarity to targets. * p < 0.05, ** p < 0.01 


hippocampus. Void of semantic information organized in varying hierarchical cate- 
gorical structures, it eliminates the variance in retention performance due to varying 
amount, type, and linkage of associations. Furthermore, with categorical stimuli, 
also similarity judgements are subject to a multitude of uncontrollable influences. 
A zoologist might judge the similarity of two seahorse depictions differently from a 
layman. A first tentative neuromorphic model of basic hippocampus function should 
not have to deal with high-level representations acting on the perceived similarity of 
two stimuli. This was a major reason to consider sensory stimuli. 

A second and just as important reason was the parametrical control of similarity 
by mixing sensory stimuli without loss of validity. Sensory stimuli can be mixed to 
create intermediate stimuli with any desired degree of similarity. This is of specific 
importance for studies sizing pattern completion and separation, and cannot be done 
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with categorical stimuli as provided by the MST. One cannot mix the picture of a 
four-leaf clover with the picture of an oil can to get a picture of something that is 
somewhere between a four-leaf clover and an oilcan and reliably identifiable as such. 

Sensory memory is usually reported in two variants. For instance, Cowan [48] 
describes short and long auditory stores, with short auditory stores keeping infor- 
mation for up to 300 ms, and long auditory stores retaining auditory information 
for at least several seconds. Please note that the lifetime of the so-called long stores 
corresponds to the lifetime of classic short-term memory. There is, however, ample 
evidence that sensory information may be stored for even longer periods, paralleling 
the classic findings for long-term memory (for a review see Ref. 49). In other words, 
there is no reason not to use sensory stimuli in classic memory experiments; as 
to the new paradigms that came along with the concepts of pattern separation and 
completion, sensory stimuli offer the advantage of parametric control of similarity. 

The Visual Sensory Memory Task (VSMT) assesses pattern separation and 
completion in a way similar to the MST, using sensory stimuli instead of depic- 
tions of objects. In addition, it comes with an analysis different from that presented 
by the authors of the MST: Instead of subtracting certain entries of the 3 x 3 response 
matrix, itis analyzed in terms of classic Gaussian Signal Detection Theory [50]. Gaus- 
sian SDT assumes that a monitored quantity—in our case familiarity—is distributed 
normally on a decision axis, with equal standard deviations for the different stimulus 
classes (foils, lures, repeats). Only the means differ, with repeats having the highest 
familiarity, and foils the lowest. Decisions for a specific response—in our case “old”, 
“similar”, or “new”—are made on the basis of criteria on this decision axis. The 
differences between the means of the distributions of stimulus classes with a certain 
familiarity (lures, repeats) against the stimulus class with the lowest familiarity (foils) 
are measures of the so-called sensitivity (d’). The possibility to quantify the physical 
correlation of lures with their respective targets allows to establish a psychophys- 
ical relationship between physical correlation on the one hand and psychological 
similarity expressed in d’ on the other hand. 

Figure 6a shows two examples of the stimuli used in the VSMT, with the second 
example being a lure to the first one. The stimuli are composed from grayscale visual 
pink noise, i.e., visual noise that has a 1/f distribution of spatial frequencies. The use 
of pink noise was inspired by the finding that natural images show a 1/f distribution 
of spatial frequencies, and that cortical cells are tuned to exactly this distribution of 
spatial frequencies [51]. This pink noise is then subjected to a Gaussian envelope, 
smoothening out the grayscale variations of the noise at the borders of the stimuli and 
preserving them most prominently in the center of the stimuli. This measure should 
prevent memorizing strategies based on visual artifacts occurring at the borders of 
the stimuli. 

Analyzing the lure and target sensitivities expressed as d’ values as a function of 
the physical lure-target correlation we found a psychophysical law which links the 
sensitivity to the quartic correlation of lure and target grayscale values: d’ œ rpy*, 
where rır is the correlation of lure and target. Figure 6b illustrates this result for 
two different mixing algorithms, called blending and pixel substitution. Figure 6c 
shows exemplary behavioral data, with the lure correlations already chosen such that 
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Fig. 6 The Visual Sensory Memory Task (VSMT): a Two example stimuli of the VSMT. The lower 
stimulus is a lure to the upper stimulus: the physical correlation of the grayscale values of the two 
stimuli is r? = 0.69, i.e., 69% of the variance of the grayscale values in one of the two stimuli is due 
to the variance of the grayscale values of the other stimulus. b Sensitivity of lures (r < 1) and targets 
(r = 1) as a function of lure-target correlation. If the latter is expressed as quartic correlation (r*), the 
measured sensitivities approach a linear relation. c Exemplary behavioral results of the VSMT. With 
increasing correlation between lures and targets, the frequency of the response “new” decreases, 
and the frequency of the response “old” increases. The frequency of the response “similar” follows 
a nonmonotonic course, increasing up to a maximum at about r* = 0.5 and decreasing thereafter. 
The symmetry of these results demonstrates that the quartic correlation of the physical grayscale 
values of the pink noise stimuli is a good predictor of the psychological similarity, expressed in 
sensitivity values d’ 


they are distributed equally-spaced on a quartic correlation axis in order to obtain 
equally-spaced sensitivities. 

The process of pattern separation, i.e., the differentiation between similar stimuli, 
can be examined quantitatively when using sensory stimuli, based on familiarity 
judgements. However, in order to test pattern completion one needs an indicator 
of identification. This can be realized by having the participants name the stimuli: 
Completing a fragmentary pattern can be verified if the pattern is identified by telling 
its name. 

So, we tested the ability of participants to learn names of the sensory stimuli and 
to correctly identify them immediately after learning, and after one week. Figure 7 
shows the naming performance directly after the learning, and one week later. Naming 
performance is calculated for “hits”, i.e., trials, where participants recognized the 
stimuli correctly as targets, as well as for “misses”, i.e., trials, where participants 
claimed the (old) stimuli to be new but were nevertheless requested to guess a name. 
Obviously, the naming performance is better for hits than for misses, but it is still 
well above chance level for misses. Most importantly, naming performance does not 
decrease strongly after a week. 

The latter data illustrate that sensory stimuli can be used to test the transition of 
short-term memory to long-term memory, which is the principal role of hippocampus 
in human memory. Devoid of the influence of higher-level semantic representations 
and with the additional advantage of the possibility to construct stimuli with any 
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desired degree of similarity, they provide an excellent opportunity to directly compare 
neuromorphic models of the hippocampus with human behavioral performance. 


4 Neuromorphic Investigation Pathways 


In the last subsection we have shown the special importance of pattern completion 
and pattern separation for memory formation and their significance for higher cogni- 
tive functionalities of the brain. For the emulation of these functionalities within 
neuromorphic systems, the bridge to cellular learning paradigms is important. In 
particular, this requires reconstructing cellular learning and network architectures in 
a way that ensures global functionality through local building blocks. One possible 
approach will be presented in this subsection, with special use of memristive devices. 
Based on the Hebbian learning theory, we will show how this theory can be applied 
to memristive devices and how it can be used to construct network architectures that 
enable the emulation of hippocampal learning forums. 


4.1 Hebbian Learning 


As early as 1949, the psychologist Donald Hebb suspected a temporal relationship 
between local neuronal activity and the change in synaptic connectivity there. Today, 
this relationship is known as Hebb’s learning rule and it reads as follows [52]: “When 
an axon of cell A is near enough to excite cell B or repeatedly or persistently takes 
part in firing it, some growth process or metabolic change takes place in one or both 
cells such that A’s efficiency, as one of the cells firing B, is increased.” Now often 
and somewhat more simply stated this sentence says: “neurons that fire together wire 
together.” Thus, the temporal correlation of signals determines the strengthening of 
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synaptic connections. In the original sense, this only includes learning processes, 
whereby a similar rule also applies to unlearning. An extension made to this comes 
from Stent in 1973 and reads as: “Neural connections weaken when they are inactive 
at the same time that the postsynaptic neuron is active, or they will weaken if they are 
active, but consistently fail to cause the postsynaptic cell to fire an action potential” 
[53]. 

While Hebb’s learning rule gives an intuitive approach to cellular learning mech- 
anisms, a mathematical approach is needed for a formal quantitative model descrip- 
tion. Following Ref. [54], this can be obtained via the characteristics of synaptic 
plasticity. As mentioned above, a fundamental feature of synaptic plasticity is input 
specificity. This means that the change in synaptic weight w,;; between pre-neuron 
j and post-neuron i depends only on local variables and thus only on information 
available at the synapse site. This can be formulated mathematically as follows [54]: 


dwij 


dt F (wij, Aj Aj i) 


Here the pre- and post-synaptic activities A ; or A; are given by the voltage dependent 
function Aig) = glui), where F(wij, Aj, Aj) is a function dependent on the 
learning process, which we will specify in more detail next. An important property 
of cellular learning is cooperativity. This means that the synaptic weight changes 
when neurons are active simultaneously. This allows a simple ansatz for the function 
F (wij, Aj, Aj): 


F(wij, Aj, Aj i) = aA; $ Aj 


where g is a positive constant (a > 0) named as learning rate. However, it is useful 
to make a weight-dependent, so that one excludes unlimited weight growth: 


awi) =y: (1 = wij) 


Since y is a positive constant and if w;j is normalized between zero and one, 
a weight saturation is therewith obtained. The third important property of cellular 
learning is associativity. This allows local learning rules to be transferred from 
cellular level to a multidimensional network level [54]. Furthermore, competition 
considers the limitation of shared synaptic resources, which leads to the fact that 
weights can only grow at the expense of other synaptic weights. One commonly used 
and simple possibility to implement competition is an adaptive threshold voltage (or 
sometimes referred to as a sliding threshold) for post-neuron activation [54, 55]. 


Emulation of Learning Behavior in the Hippocampus: From Memristive ... 423 


4.2 Memristive Hebbian Learning 


Neuromorphic electronics is essentially concerned with the implementation of 
biological information processing within electrical circuits and systems [1]. There- 
fore, the emulation of synaptic plasticity via electronic devices is an important aspect 
for the hardware realization of neural circuits. In this context, memristive devices 
have shown their potential in recent years and are increasingly used as substitutes for 
synapses in artificial neural networks [2]. In the following, we will show how they 
can be described using Hebbian learning theory [56]. First, however, a short formal 
presentation of the concept of memristive device is given. 

In their simplest form, memristive devices (also known as memristors) consist of a 
metal insulator metal structure as sketched in Fig. 8a. The insulator is the memristive 
layer which changes its resistance when an electrical voltage is applied. Thus, this 
device has a memory effect for electrical signals, which also explains the name as 
memory resistor. The idea of memristive devices goes back to Leon Chua, who 
first formally described the class of electronic devices within a mathematical theory 
[57]. Following Chua, the current-voltage characteristics (Z-V curve) of memristive 
devices can be described by the following set of differential equations: 


d 
1=GG,V,1)- Vwith— = f(x, V, t), 


where x is called a state variable and ranges between zero and one, while f(x,V,t) is 
a function describing the state dynamics under external voltage stimuli. The conduc- 
tance G(x, V, t) is named as memductance and can be linked to the state variable 
via [58] 


G=x.- Gon + ad = x): Goff- 
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Fig. 8 Emulating synaptic plasticity with memristive devices: a schematic representation of a 
memnristive device for emulating synaptic functionality. The change in atomic structure in the 
memristive layer results in a change in the resistance value of the device. b Emulation of long- 
term potentiation (LTP): after induction of LTP, there is a long-lasting increased depolarization of 
the post synaptic potential (PSP) upon activity of the pre neuron. This can be reproduced by the 
bifurcation of filamentary conducting structures within a memristive device. (Adapted from [56]) 
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Here Gon and Gosp are the maximum and minimum conductance of the device, 
respectively. In 2007, scientists at HP labs made a connection between this theory, 
developed by Chua in the 1970s, and the resistive switching properties of resistive 
materials [59]. In this class of materials, a change in the local atomic configuration 
results in a change in the resistance of the devices [2]. Thus, the function f (x,V,t) 
describes the voltage-driven atomic reorganization within the memristive layer [59]. 
The emulation of synaptic plasticity using memristive devices is shown in Fig. 8b. 
When a high electrical voltage is applied, the atomic configuration within the memris- 
tive layer changes. Shown here is the injection of metal atoms into the insulating 
memristive layer building a metallic bridge between the two electrodes of the device. 
This leads to a permanent change in the resistance of the device and can emulate long- 
term potentiation (LTP). Furthermore, by applying a sufficiently negative voltage, 
the metallic bridge can be broken. This increases the resistance of the memristive 
device again, which can be identified as a long-term depression (LTD). A special 
property of memristive devices is that they also have gradual resistance changes, 
which can be used to simulate Hebb’s learning. Thus, using the Hebbian learning 
theory and the Memristor equation, the following relationship can be identified: 


dw; dXij ra 1 dGij _ 7 7 
di a GaGa di = f(x g(w;),t) 


Thus, synaptic plasticity can be simulated via the voltage-driven resistance switching 
mechanism in memristive devices. Here, f (xij g(uij), t) represents a voltage- 
dependent function that describes the voltage curve across the memristive device. 
It was shown that for memristive devices this equation can be identified with the 
logistic equation [60]: 


F (xij, gluij), t) = B(xij, gluij), t) x17 O(1 — xij) 


It is important to note that this equation has a state and voltage dependent learning 
rate 8 and therefore differs from the normal logistic regression. This takes account 
of the special properties of memristive devices, whose functional mechanisms are 
usually based on ionic processes and have threshold properties. Thus, a change of 
the resistance of the device depends on the duration and amplitude of a voltage 
stimulus, as well as on the current resistance state [60]. Although the specific form 
of 8 depends on the type of the memristive device and its physical properties, some 
important statements can already be made at an abstract (descriptive) level: (i) in the 
simplest case, the voltage function g(u ; j) can be represented by the voltage drop 
across the memristive device, i.e. at constant potential at the one electrode via the 
voltage u; at the opposite electrode. (ii) Following reference [60], the learning rate 
is linearly dependent on the state variable (8 = yx;ju j), so that the memristive state 
change for the emulation of synaptic learning processes in the framework of the 
Hebbian learning theory can be summarized as. 
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dxij 
Pa = (1 — xi) xij? -uj 
Thus, we have a compact model for a memristive device within the framework of 
Hebbian learning theory. Figure 9a shows the I-V curve obtained for sweeping the 
voltage u; between 7 V and —/ V (see arrows) with that model. Therein, a state- 
dependent learning rate 8 (u j) (red curve) is compared with a state-independent 
learning rate 8 (xi ju i) (black curve). What can be seen is that by considering the 
state variables in the learning rate, an asymmetry between the set process (at positive 
voltage) and the reset process (at negative voltage) is obtained. In addition, the 
state variable dependent characteristic leads to a stronger non-linearity. For better 
visualization, the change of the state variable x is depicted in Fig. 9b and the function 
f is plotted in Fig. 9c. One sees a clearly stronger non-linear behavior of a state 
dependent learning rate (black curves) over the time course of the voltage change, 
which agrees with the threshold behavior of real memristive devices. It is shown in 
[4] how this model can be adapted to real device characteristics by choosing suitable 
representations for B(x; a g(u i jb t). In the following, however, we will discuss the 
biological plausibility of the model in more detail and show how it can be used to 
reproduce synaptic plasticity. 


a 40°! bo; 
[ x 0.5 
104 } 
= e | 0 10 20 30 40 
= 10 F Reset | j time (arb) 
D Dees | 
= al A i C 10 : > 
O 10 E Set 10° 
t j | 107 i 
107; 10° 
| 108 
vn an 0 10 20 30 40 
-1 -0.5 0 05 1 time (arb) 
dx; 
Voltage (V) — Stay ay) xy-y Liyan 


Fig. 9 Phenomenological model of a memristive device with a state-dependent learning rate B(u j) 
(red curve) and a state-independent learning rate 8 (xij u j) (black curve). a I-V curve for a voltage 
sweep between 1 V and -1 V. b Change of the corresponding state variables. (c) Comparison of the 
two functions used 
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4.3 Emulation of Synaptic Learning 


To show the biological plausibility of the memristive Hebbian learning model, we 
will follow reference [6] and first consider the emulation of LTP and LTD. For 
this purpose, the voltage clamp method was emulated first and compared with 
physiological data obtained from hippocampal CA1 neurons. The results obtained 
by the memristive learning model are shown in Fig. 10. For this purpose, the post- 
synaptic potential was used as a constant voltage offset (u; = Vpost), while a voltage 
pulse train was applied to the pre-neuron side (cf. Figure 10a). As sketched in Fig. 10b, 
this causes the resistance state of the memristive device to be set or reset depending 
on the post-synaptic offset Vposr. Here, use; is to symbolize the critical voltage for 
changing the device state. Therewith, the post-synaptic potential determines whether 
the resistance of the memristive cell is increased (emulation of LTD) or decreased 
(emulation of LTD) in a good agreement with physiological data of [61], as shown 
in Fig. 9c. 

An important property of synaptic learning is the temporal correlation between 
neuronal activities on the pre-synaptic side in relation to the post-synaptic side. In 
this context, spiking neural networks (SNNs) have been shown to mimic information 
processing in biological neural networks very well. A central component of infor- 
mation coding in SNN is provided by the neurons of the network, whose activity is 
coded in spike patterns (voltage pulse trains). The strength of a stimulus is translated 
into a number of spikes. Therefore, the all-or-nothing principle is used, which means 
that if the input stimulus (which can be represented by a current J;,) depolarizes the 
post-synaptic potential u; to such an extent that it exceeds a critical threshold O;hres, 
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Fig. 10 Emulation of the voltage clamp method for measuring long-term potentiation and long- 
term depression with the memristive learning model. a Schematic representation of the voltage 
function over the memristive device. b For the simulation, the post-synaptic contact was used as the 
voltage offset, while a voltage was applied to the pre-synaptic connection. use; describes the required 
set voltage of the memristive device. c The change in resistance value obtained with the model as 
a function of post-synaptic voltage (solid line) compared to data from hippocampal neurons from 
the CA1 region. (Adapted from [6]) 
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a short voltage pulse (spike) is generated. Depending on the stimulus strength, more 
or fewer spikes are generated. Thus, a high activity of the neuron can be identified 
with a high number of spikes, i.e. with a higher frequency than in comparison to 
a weak input current, which generates only a few spikes. Mathematically, this can 
be represented quite elegantly within the framework of the leaky-integrate-and-fire 
neuron model which is generally given by the following equation: 


a = F(uj) +ô- Lint) 
Here F (u j) is a function that describes the voltage integration and must be defined 
in more detail and 6 is a constant. For the determination of F (u j) there are several 
possibilities, whereby [54] gives a good overview. In the context of the consideration 
made here we use the quadratic leaky-integrate-and-fire (QIF) neuron model specified 
by the following expression: 


gm 


d gi (uj — ur) (uj — ue) + Iin (t) 


where gz is a constant that has the dimension conductivity per voltage, C is the 
membrane capacitance, u, is the resting potential, and u, is the threshold potential 
for self-induced spiking of the neuron. A schematic representation of the generated 
voltage curve u;(t) of the QIF neuron model is shown in Fig. lla. Here, a spike 
signal from neuron A or B (orange or black) is always transmitted to the network 
as soon as uj exceeds the threshold value 6;,¢;. In the model used in reference 6, 
the neuron model was also extended in such a way that when one of the neurons in 
the small network of two neurons sketched in Fig. 11a spikes, the current value of 
the respective other neuron is used for the offset potential V,,;;. Then the synaptic 
weight can be increased (LTP) or decreased (LTD) from the respective value u ;(t) 
of the other neuron. 

As already mentioned, the temporal order of the spikes is of relevance for SNN. 
An important learning schema in this context is spike-timing-dependent plas- 
ticity (STDP), which is sketched in Fig. 11b. STDP is an asymmetric form of 
Hebbian learning and allows causality in undirected neuronal networks. It speci- 
fies the strengthening of the synaptic connection (potentiation) if the pre-neuron is 
active before the post-neuron and a reduction of the synaptic connection strength 
(depression) if the post-neuron was active before the pre-neuron (cf. Fig. 11b). If 
neuron A is chosen as the pre-neuron and neuron B as the post-neuron, SDTP can 
be simulated in this network using the memristive Hebbian learning rule as shown in 
Fig. 11b. Here, the parameters of the model were adjusted to show good agreement 
with experimental data of Bi and Poo [61]. However, the temporal ordering of STDP, 
which leads to unidirectional connectivity in networks, is not valid for the complete 
frequency range. For spike patterns at frequencies above 40 Hz, the temporal order 
of spikes from pre- and post-synaptic neurons is cancelled as shown in Fig. | 1c 
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Fig. 11 Emulation of spike-timing-dependent plasticity (SDTP). a Schematic and time course of a 
network of two leaky-integrate and fire neurons for emulation of STDP. When one neuron fires, the 
respective voltage value of the other neuron is used to change the resistance state of the memristive 
devices coupling the two neurons (represented by the gray arrows). Here u; describes the threshold 
between setting and resetting the memristive device b schematic representation of the spike order 
described by the STDP and the SDTP curve obtained with the model (red line) compared to measured 
data (dots). c Dependence of the spike order on the frequency of the spike pairs. The solid lines are 
results from the simulation, while the dots are measured values of real neurons. Here, in the right 
image, the model of the memristive devices was extended by a leakage rate, which leads to a better 
match at low frequencies. (The figure was adapted from [6]) 


for the example of cortical pyramidal neurons (red and blue data points) [62]. To 
reproduce this behavior in the model, a fixed time delay of +10 ms between the 
spike trains (A and B) was chosen, so that the individual spike pairings are constant 
with respect to each other. Hereby two cases were considered in reference [6]: (i) 
the resistance state is constant between two following voltage pulses and (ii) the 
resistance value increases between those pulses as a result of a continuous reset 
mechanism typically found in ionic memristive devices [4]. As a result, Fig. 1 1c (red 
and blue curves) shows that within the model the transition between asymmetric and 
symmetric behavior can be well emulated. The latter is especially relevant for the 
formation of bidirectional connections in networks as we will see in the next chapter. 
Furthermore, it can be stated that a low and autonomous reset of the device resistant 
state leads to an improved match at low frequencies of less than 10 Hz. 


4.4 Emulation of Network Dependent Learning Schemes 


Learning and memory formation in biological networks is closely linked to network 
topology [63]. Thus, it is the connectivity between individual neurons that determines 
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how a network responds to external stimuli and thus how it processes the information. 
While in machine learning the topology is fixed in the form of neuronal layers and 
the connectivity between the layers is determined by training processes, in biological 
neural networks a large part of the topology is formed by the learning process itself. 
In this process, connections are formed between networks that did not exist before 
and thus the network grows with its task. 

Emulating growing networks within electronic circuits is difficult to realize in 
hardware, but all-to-all connecting networks, that are, networks in which each neuron 
is connected to every other neuron provide a simple opportunity to come a step 
closer to study the development of the topology of biological networks. A good 
example of this are the DG and CA3 regions of the hippocampus, which act as 
biological role models of the mentioned essential tasks pattern completion and pattern 
separation. This is considered to be the basis of associative memory in particular and is 
characterized by incoming information being stored sequentially and independently 
of each other. Thus, stored information can be retrieved and restored even if it was 
only partially presented. In the network topology of the hippocampus, recurrent 
connections in the DG and CA3 region could be identified for this purpose, forming 
an auto-associative network. This can be well modeled by an all-to-all network [6]. 

The memristive network model of Ref. [6] which emulates those two key func- 
tionalities, i.e. sequential learning and pattern completion, in form of an all-to-all 
network is shown in Fig. 12. Therefore, the memristive learning model was used 
and external information was presented to the network in two different ways: (i) 
by applying the individual pixels of visual patterns in parallel and (ii) by applying 
the individual pixels of a visual pattern sequentially. A schematic representation for 
better explanation is shown in Fig. 12a. The obtained connectivity matrices for these 
two cases are shown for pattern completion and sequential learning in Fig. 12b and c, 
respectively. For pattern completion, it was shown that using the memristive learning 
model, there is an increase in bidirectional connectivity at higher frequencies, which 
can be explained by the behavior shown in Fig. 12c. A unidirectional connectivity 
pattern is the result in the case of a sequential pattern presentation during the learning 
process (see Fig. 12c). 

This allows the topology of the network to be shaped depending on the presen- 
tation of the input data. The latter is particularly important for learning in biolog- 
ical networks, since it allows for experience-dependent learning, i.e., depending on 
how information is presented (experiences have been made), the topology of the 
network and thus the functionality of the neural network changes. Thus, memories 
can shape the performance of the system. This is an important feature related to 
episodic memory, in which events are linked by their temporal sequence. 
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Fig. 12 Memristive network model of hippocampal structure to emulate pattern completion and 
sequential learning in the context of an all-to-all network. a Test pattern and network topology. 
The way a pattern is presented to the network gives rise to different network topologies. b Connec- 
tivity matrix of the network after learning for different repetition frequencies in the case of pattern 
completion (pattern A and A’). c Preserving connectivity matrix for the sequential creation of pixels 
of the test pattern during learning. (Adapted from [6]) 
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